maven es más que un sistema de repositorios, ya que tambien permite construir los proyectos,  compilándolos y generando los artefactos necesarios en cada caso. Esto puede resultar extraño para alguien que viene de un mundo en el que los lenguajes no tienen que ser compilados,  en python todos los requerimientos son de tiempo de ejecución, sin embargo, en un lenguaje compilado, pueden existir distintos tipos de dependencias, lo que hace casi una necesidad que el mismo software que se usa para gestionar las librerías sea usado para construir el proyecto.

Una de las cosas que más me gusta de maven es el famoso “maven war overlay”. Básicamente es un mecanismo que permite construir proyectos y despliegues de una manera similar al paradigma de la herencia. En el proyecto en el que estoy trabajando actualmente usamos este mecanismo para modificar por partes los distintos aspectos que deseamos personalizar, como el aspecto, algunas extensiones o los parametros necesarios para ejecutarlo en producción o pre-poducción.

Se puede usar maven desde eclipse via un plugin. Esto es realmente deseable para gestionar el pom.xml, ya que cuenta con un editor hecho para humanos en lugar de tener que trabajar con un larguísimo fichero xml. Me parece particularmente útil el mecanismo para añadir dependencias ya que se encarga de buscar el groupid por ti lo cual es altamente cómodo.

Despite my residual developer role, these days I have been concerning about the Jasig CAS upgrade from and old version. Jasig CAS use the maven war overlay method to deal with the customization, and that’s great. We have a new artifact using a custom skin, custom theme, xml configuration files with the correct data source and so on. The upgrade process should be simple, but the new CAS version use a new version of spring, so I should update (Do I really need to update all that xml reference manually?) lots of xml files with the new reference. They also decide to change some components so I need to include some new configurations too. And all this try-and-error process follows the next sequence:

• Modify some files
• mvn clean package to generate the war
• Deploy the war in Tomcat
• Visit the site. If something went wrong try to figure out (through the trace of Tomcat)

Therefore the process is quite slow, as it include some slow steps such as mvn and deployment. I still don’t feel comfortable with the java process. I come from dynamic languages such as python (mainly Django) and, from my view, java projects needs more hardware resource, and developing them is slower too, so I think I’m missing something about that kind of projects.

Even so, I feel slower than usual but I do my work as well as I can so feel great when get home.

The first project I look is the one I consider more exciting. To make a long story short, it involves two pieces of code.

The first one is called “salmorejo” (a typical canarian food is “conejo en salmorejo” or in english “rabbit in salmorejo” where salmorejo is a sauce). salmorejo use rabbitmq to spread some messages over a wide variety of systems. For instance, you can create a new user in and application completly uncouple to this and, throw salmorejo, the addition of the new user to ldap and the creation of the mailbox magically occurs in two different servers. salmorejo is a big black box to me so far, it’s my first experience with rabbitmq or any AMQP so it’s not a big surprise.

The other one is called “gestion de usuarios” (user management). This is a Django application which has access to several data origins (the databases used by the corporate applications in my organization), and to the queue system. The multidb support in Django is awesome, it makes possible to use this application as clue in the identity management process. The Django modularization is also awesome,it makes possible to have several small sub-projects to work with. As I feel comfortable with Django I feel much better working with this sub-project, and, in just one week, there are some code written by me in production. The lack of tests is disturbing in a long term project like this, but there are some tests over there (I hope the most critical parts ). I agreed with my boss to provide tests for all new code.

On the other hand I had some time with Jasig-CAS. This projects seems to be much more simple. We already have an installation using and old version. I’m supposed to deploy the new version. I had a first contact with maven. The maven overlay system for CAS is really smart, although I had not success deploying the war I generated. I feel the cas project could be closed in a couple of weeks. We are going to try some new TicketRegistry (we are using JPA and want to try Memcached and possibly Ehcache before going into production).

The minor projects, those projects out there that used to break my concentration, eats lot of time. I also feel some lack of team work, in my last position every single day had some pair working session, but, in the new one, everyone in the team work in a different thing therefore there are almost no pair working. So far I’m learning tons of things about services, application servers, oracle databases, etc.

I feel exited about my new position because of the self developed project. I will be the project leader in some weeks. It’s a recently launched software with tons a changes to be done and not a single test, but it’s also my closer chance to be a full time python developer. I feel a little bit afraid too, because all the work done before me.

I’m also leaving Galotecnia. Some years ago, some others and me found Galotecnia, a really small company focus in sysadmins, django development, timing race and others. Now is my time to leave. It was an interesting experience, but I give up. I can’t work in two different companies every single day, with that many context switch, and that many hours a day.

You can find the code in github.

At this time the code there can do next things:
* Generate combinations with the numbers and stars provided
* Filter them with the registered filters
* It has a filter system flexible enough to be able to include more filters
* It has some examples filters such as evens, odds, tens, highs, lows.
* This is currently for developers only (no gui, etc.)

Next step is provide a check method that, given a combination and a winner, calculates how much money the combination would earn

Let’s see it

for filter in self.filters:
            f = filter['function']
            args = filter['args']
            kwargs = filter['kwargs']
            valid_combs = itertools.ifilter(lambda x: f(x, *args, **kwargs), valid_combs)

In this code self.filters hold a list of dicts. This is part of the pluggable filtering system in pyeuromillions, but the key line is last. ifilter method return all items in seq that pass a test (in this case the lambda function). You can use ifilterfalse in a similar way but it’d return all items failing a test.

It performance really good. In less than 10 seconds it generates all combinations of 5 numbers between 1 and 50 having 3 even numbers in my actual computer (which is not a supercomputer).

Next time I spent some time writing pyeuromillions code I’ll focus in create a set of filtering functions.

Euromillions is quite simple. There are numbers and stars. A combination is valid if it contains 5 numbers (1,50) and 2 stars (1,9). In this first post, I’ll focus in, given a list of numbers and stars, generate all valid combinations with them.

The first step is to make all the available combinations with the numbers. Let’s assume self.numbers has a list of numbers between 1 and 50:

import itertools

valid_number_combs = itertools.combinations(self.numbers, 5)

Now we should do the same with stars. It’s pretty the same

import itertools

valid_number_stars = itertools.combinations(self.stars, 2)

Finally we should combine them using a cartesian product

valid_combs = itertools.product(valid_number_combs, valid_stars_combs)

And that’s all (folks). It’s pretty simple, but perfomance it’s great. In my personal computer it takes just 9 seconds to generate all available combinations with all numbers (1,50) and all stars (1,9). There are 76.275.360

Visit the github of the project for the code.

In the next post I’ll focus in provide high level filters such as limiting the number of odd/even numbers allowed in the same combination.

Isolating each project makes them fully independent. You can install different versions of Django, and any other lib you need using easy_install or pip making it really simple. You can even has several virtualenvs for the same project (this could be interesting to try new versions of libs). You can upgrade your libs in a simple manner.

Among disadvantages maybe the more clear is that, if you use almost the same libs in your projects you will be having several copies of the same things, and even worse, as they are in differents path you will have several copies of the same things in memory.

IMHO the servers are quite cheaps to try to optimize things to that level. Having a dedicated server in ISPs such as OVH or 1AND1 is cheap. Let’s said your choiced plan is about 750 euros/year, this means about 62.5 euros/month. Having this in mind (and assuming the projects I meaning are those that can be run several in a server) you should note that if deploying your projects for performance makes you work (let says) 4 hours more in a month than deploying them in a easy way, you would optimize your time contracting another server.

That’s way I’d rather to use an easy way to deploy projects even if the mem footprint is higher.

Now is time to think in the future, we have expend some time looking for the state of the art of portable apps such as django-photologue, django-cms, Dinette, socialauth among others. It’s great to see how portable apps are making Django even better, as you can plug some well-tested application into your app and get those features most sites need.

At this time we have integrated django-photologue (not in production environment yet) in Olymper.com. We are planning to include some other apps in the next months as well.

Django-photologue has some really nice features. The way it manage sizes is just outstanding. It’s really customizable and flexible while still easy. The integration is really easy just plug, write some templates, link from the other applications and start fun!!!.

I expect to be less busy for the next months so I’ll try to write something from time to time.