Help support Fleet-Up
Not logged in
Home    News    Contact    About    API    Donate     Fleet-Up on Twitter Fleet-Up on Facebook
Home // News
June, 13th 2014 - Tune Up
It's been a long time since I released - nearly two years. Since the release I've put work into extending the features and improving what is there, mostly at users' request. However, there are a few core processes that have been lurking around since the beginning and one or two of them have been acting like a horse's ass recently. I mean, to be fair to myself, many of the background processes have outperformed my initial expectation. But there comes a time in the life of software when it's time to crack open the hood and give it a good old tune-up.
The Guilty Party
So, what's been playing up? Primarily, the hoard of calculations that are involved in telling you what ships you can fly and at what level, the core of Fleet-Up's functionality, has out grown it's use. Regular Fleet-Up users may have noticed over the past weeks there have been times when pages are not loading, particularly in the areas of skills and flyable information. Often these are temporary, however, on a couple of occasions there have been more extensive times of interruption.
I'd like to take a moment to apologise to anyone who's experienced issues of this nature. I have been aware of them and I've wanted to improve things for a while but as with any pet project it's all about finding the time and motivation. As it happens around this time an unexpected party asked me a question on twitter about a feature I really liked the sound of - only I needed to improve things before I could put it in. So I cracked the hood and got to work.
Tuning Up
The crux of the issue I had was common among databases of size; large tables with lots of data were taking a long time to process and as a result I was seeing large table-locks that were holding up a whole bunch of user interface operations. You see, turns out that more people started using Fleet-Up than I expected and the code I wrote in this area just doesn't cut it as things scale up.
I've had a few attempts at solving the performance issues but there are a lot of factors affecting the problem. I'm looking at a large volume of data that also is fairly mutable. Skills update all the time, people join, people leave, fittings are added and removed - all the time Fleet-Up has to keep up with this and present and up-to-date view to the user.
I tried breaking operations into batches, breaking up tables, slowing down processing intervals, etc but none of that really worked. In the end I implemented two specific changes. Firstly I changed the transaction isolation for the user interface - this is something I had been reluctant to do historically but when I looked at the actual data that was being committed and/or locked, the user was not really gaining any more information by being forced to wait. So, ignoring uncommited data gives me part of the solution. But this only solved half of the issue. I also have operations occurring all the time that require table or page locks. I also needed to work out how to deal with that.
There's lots of ways of dealing with table locks on large data sets and each one depends on the specific requirements of the situation. In my case I have a large volume of mutable data and after trying a bunch of complex solutions I realised I could use the magicians trick. And, it appears to be working fairly well so far.
What comes out of this?
Well, what comes out of this is two things. Firstly I can start looking back at the Fleet-Up certificates feature that I had part-written for version 1.2 but had to abandon for time reasons but also because I was not sure the existing back end systems can support it. Secondly, I have had some cool feature requests, as I mentioned, in the fittings area that I'd like to focus on next.
Questions or Problems?
If you have any questions or see any problems please get in contact.
 Posted by Wacktopia on 13th Jun 2014