Jenkins and GIT

September 26, 2012

Jenkins is a continuos integration software designed to run tests and compile projects and providing lots of meaningful metrics over test and compilation process such as test coverage, coding style and such things. Among other features it’s possible to integrate it with several VCS, to ensure you can test every new piece of code.

Few weeks ago, I migrated the repository of the software project I’m most involved from Subversion to git. To make a long story short, the reasons were mainly the branches support. Now I need to make Jenkins use the git repository instead of Subversion one. This is pretty simple.

In my installation, Jenkins runs under its own user, which is jenkins. So first of all you need to become that user and create a new ssh key for it.

 esauro@legolas:~$ sudo su
[sudo] password for esauro:
root@legolas:/home/esauro# su - jenkins
$ $ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/var/www/.ssh/id_rsa): ~jenkins/.ssh/git
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in ~jenkins/.ssh/git.
Your public key has been saved in ~jenkins/.ssh/git.pub.
...
 

They passphrase should be empty. The next step is to give read access for the repository to that ssh key. That’s beyond the scope of this post, as it depends on the git server that you are using.

Add and ssh config file which use that key for the git server. Edit the file ~jenkins/.ssh/config and add the next

Host <put_your_git_host_here>
IdentityFile    ~jenkins/.ssh/git

You need to add the git host to the known hosts file. The simplest way to do this is just connecting throw ssh using the jenkins user. That way we also test the ssh key is working, and the ssh key is in the authorized keys on the server side.

We are almost done, just config a user.email and a user.name

 jenkins@ubuntu:/tmp$ git config --global user.email "jenkins@your.server.com"
 jenkins@ubuntu:/tmp$ git config --global user.name "Jenkins CI"

The last step is to setup that repository in your jenkin task.

Advertisements

Oracle RMAN and archivelog

September 19, 2012

As usual, I start saying I’m not a DBA, but sometimes perform some DBA task, so…

I had again a problem with a database server running out of disk space due to Oracle database usage. This time it was archivelog and RMAN. I’m pretty sure you could find information of archivelog and rman out there, so I won’t repeat it here. In order to be able to uses RMAN a database must be in archivelog mode. In archivelog mode, the redo logs, are archived in separate files instead of overwritten. This actually means that, under some circumstances your disk will be fully used.

Our knowledge base says that, if you need to free disk space, you must do this:


RMAN> DELETE OBSOLETE;
RMAN> CROSSCHECK ARCHIVELOG ALL;

If you don’t get enough space after that, then


RMAN> BACKUP ARCHIVELOG ALL;
RMAN> DELETE ARCHIVELOG ALL;

However, in my experience what it actually happens when you do this is:

  • RMAN creates a new backupset for archivelogs
  • The archivelog files are removed

So it doesn’t free any space if you don’t move the backupset outside the filesystem. Even worse, it actually uses more disk space until you delete archivelog.

After removing (lot of times) archivelogs and RMAN backupset, the problem was still there, because new archivelogs files appeared, so, as it’s a critical database, and we still have export backups, we decided to put noarchivelog mode, until we realize what’s going on with this database.

More information about archivelog and RMAN:


Oracle ADR

September 9, 2012

Although I’m not a DBA, from time to time I do some DBA tasks. I decided to include them here.

Last friday, I got an alert of disk usage from an Oracle database server. As usual I started using the command “du” to trace the problem to the directory causing it. And I found the directory was called diag. After that I looked inside and found lots of trc files. After some google I knew it’s a new feature in 11G called Automatic Diagnostic Repository. Every time Oracle database has a problem, meaning returning some error, it stores a dump and some log files regarding the incident. The default policy is quite conservative, it stores traces for a whole year, while dump files for 30 days

To manage these files you should do it from a tool called “adrci”. Here is how I did the purge, after having a look to the policy used.

adrci> show control

ADR Home = <home_path>:
*************************************************************************
ADRID                SHORTP_POLICY        LONGP_POLICY         LAST_MOD_TIME                            LAST_AUTOPRG_TIME                        LAST_MANUPRG_TIME                        ADRDIR_VERSION       ADRSCHM_VERSION      ADRSCHMV_SUMMARY     ADRALERT_VERSION     CREATE_TIME                             
-------------------- -------------------- -------------------- ---------------------------------------- ---------------------------------------- ---------------------------------------- -------------------- -------------------- -------------------- -------------------- ----------------------------------------
2741315226           720                  8760                 2012-03-27 08:25:55.733126 +01:00        2012-09-05 00:13:23.940809 +01:00                                                 1                    2                    80                   1                    2012-03-27 08:25:55.733126 +01:00      
1 rows fetched

adrci> purge

</code>

Vacations over, Jasig CAS and Django

September 7, 2012

This is my first week after a whole month on vacations, so I’m feeling plenty of energy to work in my two principal projects at work:

  • Jasig CAS
  • A Django app to manage digital identity

On the Jasig CAS side, I have some upgrades to do, and also some challenges to face. Jasig CAS has a problem with the documentation. It symply doesn’t exists. The documentation is maintained separated from source, and they don’t have any QA process on it. I manage a custom deployment with a custom theme, and custom authentication handlers and repositories, to support spanish DNIe. To do this I need a custom login-webflow, so every new versión I need to figure out what changed to incorporated it into our login-webflow. With the source in Github the best  way to do this is using “history” on the login-webflow of the project.

I often need to expose user data through Web Services to other apps. Depending on the nature of the user data exposed this has some security concerns. My challenge regarding Jasig CAS is to include in the attributes some ones gathered throw web services. Doing this way I ensure the user data are exposed when the user is logged, so I have a point, a crazy developer won’t be able to poll the entire user database.

On the other hand the Django project has tons of things to do. The upgrade work is a continuous task, because I use so many libraries, that they are always getting security upgrades. This project is being developed in an internal svn repository. I want to move the project to git, mainly because I want to use feature branches to avoid hot fixes issues. I’m using jenkins to tests the test, although our test coverage is not that high now. I would like to publish this project in github too,  my boss used to agreed me, but he always said we need to improve the project to publish it “finished”. I convinced him that developing in github is a good idea, because if no one gets involve then I won’t be slower, but if some one gets involve I’ll expend some time in managing his/her collaborations, but also won’t expend time actually doing what he/she commited.