February 5, 2019
In the previous post, we provided a short introduction to iGene and its history, this time we’re looking into some of the challenges we’ve encountered growing a large software product.
But first, here’s a brief overview of iGene’s architecture.
Initially, this was a productive and pleasant development environment, build times hovered around 10 seconds. A developer could hit refresh in a browser and 2 seconds later see their changes – all good.
Times have changed, now build times for some of our developers are around 16 minutes and refreshing via the browser – if it works – takes minutes. The result is a slow, frustrating and unproductive development environment.
The reason for this slowdown in development is massively increased code size. A quick run of cloc shows that over the last six years, our code base has grown (over) six-fold. Looking at actual lines of code, in 2012 iGene had 362,366 code lines and now it has 2,200,415!
Over the years we’ve tried a number of solutions to the problem. The items of note were:
We moved from Maven to Gradle to take advantage of the incremental compiles and a few other tricks it allows.
This helped to a degree, although in honesty we’ve not kept as tight an eye on build issues as we should have, it’s still better than maven. More effort on our part is needed to make it silky smooth.
We use to use JRebel to avoid restarting the application. This worked fairly well, but we ended up with many issues with this.
In fairness to JRebel, they did offer to help resolve the issues with us. However, at the time, the whole team were more than busy with customer demands to take time out to resolve the issues.
Improving this has been consistently the easiest way to improve things – i.e. throw faster hardware at the problem by using the fastest MacBooks money can buy.
Over the last year, our team has investigated a number of solutions to our predicament. In the next instalment, we’ll cover our path forward…..see you then.
January 31, 2019
Here at Genial, we’ve been developing software for Clinical Genetics and Laboratory Genetics data management applications for the last 18 years. Over that time we’ve built and supported four generations of Genetic LIMS, two of which are still in routine production use and as such are being actively developed on an ongoing basis.
Around 2006, discussions began here on a new GLIMS framework based on the experience of previous years and to address shortcomings of the previous incarnations. Designs began in the summer of 2007 with prototypes of the new system in full swing by the end of 2007.
Until that time, most GLIMS silo-ed each genetic discipline into their own modules, and per site customisation was either not possible, or it lead to different code bases for each customer. The new system’s test-based approach, coupled with the ability to allow site customization from a single code base reduced the pain of upgrades and was a new and novel approach.
What we now know as iGene began life in May 2008 and was first deployed live to a site in April 2010. iGene has been in active development over the last eleven years by the same team of developers that began it, although our team has expanded since then.
Over the last decade a lot has happened….changes in build systems, source code repositories, IDE’s and core library changes and a massive growth in required (and delivered!) functionality.
Like most large projects that survive the passage of time, iGene has picked up an amount of Technical Debt. Technical Debt – for those who are unfamiliar with the term – simply means areas that ideally require work but that have not yet been addressed.
The series of blog posts that follow, we’ll be documenting the latest journey as we work to modernise our underlying framework and prepare ourselves for the next decade to come.
Below are the next few planned posts:
September 3, 2013
This is the final part of the 3 articles discussing the use of the pedigree drawing tool. Previously, we have covered the creation of a basic diagram using the application and then added different relationships to display a complete family diagram.
With this final article, we are going to cover adding further details to the diagram such as genetic test results and personal details to individuals. Finally, we will complete this with a full video demonstration on creating a pedigree diagram using the application.
It is important to add information to the diagram such as names, birth and death dates as well as marking individuals as deceased. In order to add this information, just like changing a relationship, if you double click an individual a pop-up will appear which has the different items you can complete.
The screen shown is split into 2 tabs, these being information and genetic tests. On the information screen you can enter the relevant details which will appear under the individual.
The option to add genetic test information is on the second tab. When you first click to enter the option screen, you firstly need to enter a result and the same result can then be added to other family members on the same diagram by ticking the box next to the test. For this example, we are going to use Duchenne muscular dystrophy (DMD) which is a genetic disorder on the male side of a family.
To start with, we click the plus button and enter DMD into the text box and click on the tick to apply it to the individual. When we click close, the individual is then filled in with a colour. You can change this colour by clicking the coloured square in the top left of the diagram (which is also the legend for the chart), you can select a different colour or you can use shading if you do not have a colour printer to distinguish between the different items listed on the diagram.
With this set, we can then select another affected individual on the diagram and assign the same disease to them by double clicking on a different individual, and putting a tick in the box next to the disease. After clicking close, this will then apply the same colour/shading to the individual so it is visually obvious that they have a common disorder.
Our pedigree drawing now looks like this:
Hopefully, you should now be able to see how simple it is to create complex charts within the application and you can submit some ideas for us to work on to improve the application in the future. As an example of a more complex chart which has a family with 3 different types of cancer between them we have prepared a complete diagram below.
In order to see how quick and easy it is to create a pedigree diagram we have created a real time video of this process for a family including all diagnoses which is available for you to view.
Hopefully you now have an understanding of how to use the iGene pedigree drawing application and can see how useful it would be in your daily working schedule. If you would like to take part in our beta trial then please click the link and request an invite.
July 22, 2013
We continue to develop the pedigree drawing application and have had some excellent feedback, so thank you very much for this information. Please note the application now works in IE7 and above without the requirement for the chomeframe plugin.
Over the next few posts we are going to cover the basics of using the pedigree drawing application in order to give everyone a better understanding of how the product functions. This will also give you all the chance to pass (both positive and negative) comments and provide valuable feedback on how things should be done differently.
So, lets start at the beginning….
The first thing to notice is that the application is hosted on a test server. Since this is a beta test, we are not expecting people to put live data into the system to generate the diagrams. We would like to allow this in the future but at this stage we are trying to iron out some of the bugs.
When you first access the application you are greated by the logon screen. Since the application is currently hosted online we have implemented a quick login system with authentication codes. You will need to use a Google account or a Yahoo account to log into the application and join your access code to the account you are logged in with.
The application uses HTML 5 canvas features so a modern browser is required (Firefox, Chrome or IE9 +). It is possible to run the application in older versions of IE but this will require the use of chrome frame which will allow the application to run a lot smoother. We are working on a fix to make the application function without this plugin and we will provide an update when that is ready to be released.
Once logged in to the main screen you will be presented with the option to view your older diagrams or create a new one. From this screen you can also remove the diagrams which you no longer have an interest in or are not required.
Some of the other items on this screen are the roadmap listed on top of the page. This shows what issues and new features we are working on and the progress we have made on these, a lot of these items have already been completed but we still have further items to progress and make the whole application work the way the users want it to.
There is also the location of the feedback button where we would like you to submit further information to ourselves. This can be anything from incorrect diagrams which you have managed to produce to new feature requests or even questions on how to create different diagrams.
Now we have covered the basics of the login, some technical information and the main screen of the system, we can begin to create a new drawing. We have aimed to make the application perform most of the work with regards to layout of the items in order to save the manual drawing of links between individuals and the changes to items in the chart from taking too much time. The idea is that the system will lay things out and maintain the rules as best it can whilst keeping the generations constant on the diagrams.
Creating a new diagram is performed by clicking the Create a new Pedigree button. This then brings up the main canvas screen with a basic first relationship as shown in the screen shot. You can remove these 2 items and start again if you wish but we felt that everything will start with the basic 2 person layout. At this point you can give your pedigree drawing a title so you can distinguish it from the others at a later date.
Starting with this basic pair, we can create new items by either clicking on the line joining the individuals or on one of the individuals themselves. This will bring up different shapes in different positions. If we start by selecting the middle line which then brings 3 green buttons. A square for a male, a circle for a female and a diamond for an unknown. Clicking one of these will add a child to the relationship. A further click will add another child and you can keep going for as long as you require.
As you can see I have added 2 daughters and a son to the diagram. The application lays out the diagram for you and it will try to follow the rules of pedigree drawings as best it can (ages running left to right and the male partners on the left side of the diagram). It will also create alias items when required and show links which join generations.
In the second part of this article we will start to add further details to the diagrams including different relationships and finally adding results for tests so you can track the genetic history of patients back to find patterns or probability of disease transfer.
July 4, 2013
It’s presently difficult to look into the future and see how our iGene Pedigree Drawing tool could (or should) grow, which features it will ultimately have, and which direction(s) it could take…..ultimately of course, the answers to these questions will be provided by the users of the iGene Pedigree Draw tool rather than ourselves.
We’ve had many discussions over the last few weeks with individuals and groups and the underlying theme to these has been very positive, with extremely useful suggestions and we are extremely grateful to all concerned – you know who you are – Grazie/Merci/Danke/Thank You!
When we started we had little idea about the level of interest we would have, or the diverse ways that people would like to use the iGene Pedigree Drawing tool – we’ve been very pleased to have such useful feedback to date.
We believe flexibility will be a key factor in making the iGene Pedigree Draw tool a functionality-rich, eminently usable and must-have application, and we continue put a lot of thought into how best to enable the use of iGene Pedigree Draw in different environments, for example clinical diagnostics, screening, research and so on.
We are working towards creating an open architecture that will allow third parties to modify and extend iGene Pedigree Draw themselves. The end goal is to allow users or other software authors to extend the system. Some examples are:
The first step along this journey is to provide a simple, flexible and importantly, extendable data format for the pedigree tree, a format for which is openly available. After looking at the existing open formats available, we have decided to create our own format.
The format is JSON and it describes, by default entries on the pedigree, relationships, pregnancies and diagnoses. Below is a simple example showing Bob and Mary with their son Frank.
"children": [ "E-C" ]
Importantly, this format is extendable; for example (this is obviously an entirely hypothetical situation!) if I wanted to record the overall happiness of members in a pedigree I could simply add a happiness value, for example:
Another example might be the nicknames a person has.
"nicknames": ["Bob", "Bobby", "Robbie"]
This format allows any kind of additional information to be stored in parallel with basic patient information and allows the system to be as flexible as the user wishes it to be.
We will be producing and import and export from our basic format to other formats in the coming weeks, and will open source them so you can see how it is done.
June 20, 2013
Welcome, to this our first technical blog post!
It’s entirely appropriate that our first blog post is about communication. Describing pedigrees can be somewhat complex, especially when the pedigree contains consanguineous and multiple relationships. Pedigree diagrams allow this information to be displayed graphically and in an easy to understand format. For the last few years, we’ve planned to introduce a basic pedigree diagram solution, and we’ve finally found the motivation and time to brush off the cobwebs and turn our solution into something that we think is better than basic, in fact we think it’s amazing!
This is the first of a series covering our approach to designing our pedigree application, the design choices we’ve made along the way, the problems we’ve encountered and how we’ve overcome them.
A pedigree diagram provides a simple, visual way of displaying family members, their relationships and associated data. They are a great way of describing and showing the structure of a family and sharing that information with others. The example below shows a mother (the circle) and a father (the square) in the first generation (row) with their three children, two sons and one daughter.
This is a very simple example pedigree chart. Pedigree diagrams can often become much more complex and show far more information, as we will see in upcoming posts.
One of the very first choices we had to make is how should our pedigree drawing solution work. The choices came down to two fundamental choices:
After surveying the existing software tools available, we felt that the majority seem to favour the simpler solution, a drawing tool like Visio that is customised specifically for creating pedigree diagrams.
We thought long and hard about which solution to use. Providing a simple drawing tool would avoid the complexity of laying out pedigree diagrams correctly at the cost of burdening users with doing this by hand. Manually drawing simple pedigree charts using specialised drawing tools isn’t too difficult for experienced users, however it still takes time and effort. The larger and more complex a pedigree gets, the more time it takes to draw it manually, and the easier it is to make a mistake or misrepresent something.
Designing an application to automatically draw a pedigree chart is far more involved, but provides the following:
After some internal discussion, we decided that an automated application is the best way to go, and we started working on an application which will allow all of the above and more. Not only is our application fast and easy to use (even in it’s early beta stage), we’re told by many that it’s also a joy to use.
As you may be aware, the majority of our products are web-based applications. Web-based applications can be relatively easy to manage, access and support, so we decided to go down the same path with our pedigree drawing application. The technologies we looked at are:
It didn’t take us to long to decide on using HTML5 Canvas. After reviewing the other options, HTML 5 allowed us to function on a wide array of platforms – it even works on iPad/iPhones and Android devices
The only downside is IE 6, 7 & 8 do not support HTML 5 Canvas, however this can be overcome either using the wonderful Chrome Frame , although if for some reason we REALLY did need to to provide support for IE 7 or IE 8 we do have a trick up our sleeves and it allows for printing (More information on this in a later blog post).
So you now know what we have been working on – an awesome pedigree drawing tool base on HTML 5 canvas that provides wonderful-looking pedigree drawings, simply, quickly and efficiently.