New software release velocity

As an observer of a number of the requests that are made in this forum for tweaks to the platform, and the occasional nod to a future release, it raises an interesting question about how often software releases would be welcomed for a streamer (this is actually a topic that has seen much wider debate in the industry as more agile development methodologies are embraced across different segments, products and solutions).

Coming back to our favourite hifi components, there are clearly going to be differing views - on one hand, it is an appliance, therefore I really donā€™t want to water and feed my hifi on a regular basis with firmware upgrades and just want the thing to play music. Others may take the view that a more frequent release of potentially smaller, more incremental features as a good signal of the platform being sustained and keep those who seem to frequent this forum engaged.

Of course, underpinning all of this is code quality. Clearly this is something that everyone values. I think it arguably fair to say that releasing smaller but more often is generally good practice - but then would users be willing to test or would it just create more issues elsewhere?

By the way, this isnā€™t a criticism of the product, nor is it telling devs how to build their own code - more a question of trying to understand my own views on what is reasonable to expect.

n.b. just taking a look at the data, this is what I gather from the release notes for the Aualic Ariesā€¦

Version Date Number of days since last release
6.1.2 Build 20190903 03/09/2019 194
5.8 Build 20190221 21/02/2019 66
5.7 Build 20181217 17/12/2018 47
5.6 Build 20181031 31/10/2018 702
4.1.0 Build 20161128 28/11/2016 78
4.0.0 Build 20160911 11/09/2016 146
3.3 Build 20160418 18/04/2016 46
3.2 Build 20160303 03/03/2016 31
3.1 Build 20160201 01/02/2016 21
3 Build 20160111 11/01/2016 48
2.1 Build 20151124 24/11/2015 38
2.9.2 Build 20151017 17/10/2015 47
2.8 Build 20150831 31/08/2015 44
2.7.1 Build 20150718 18/07/2015 17
2.6.1 Build 20150701 01/07/2015 1
2.6 Build 20150630 30/06/2015 47
2.5 Build 20150514 14/05/2015 35
2.4 Build 20150409 09/04/2015 13
2.3 Build 20150327 27/03/2015 7
2.2 Build 20150320 20/03/2015 38
2.1 Build 20150210 10/02/2015 4
2 Build 20150206 06/02/2015 42
1.11 Build 20141226 26/12/2014 56
1.9 Build 20141031 31/10/2014 33
1.8.2 Build 20140928 28/09/2014 37
1.7 Build 20140822 22/08/2014 14
1.6.2 Build 20140808 08/08/2014 14
1.5 Build 20140725 25/07/2014 10
1.4 Build 20140715 15/07/2014

Interesting software archaeology. Quite some shifts in frequency. I imagine there are fascinating back-stories about this journey. Maybe @Xuanqian will write a book on it like Shiit and PS Audio did. I promise Iā€™ll read it :slight_smile:

I am happy to answer this question. Currently we are trying to increase the intervals between software release due to several reasons:

  1. Early stage firmware requires more frequently upgrade because of the bug and issues inside software, even in basic features, so they need to be fixed in a time manner. As the Lightning platform has been out for more than 5 years, it has become very stable and it is unlikely that we need a urgent update to fix some problem.

  2. We have so many features in current firmware comparing to early days, some components are relating to each other. Adding or changing any feature will require extensive testing steps and it requires a shit amount of time (sorry I say that but it is true). The testing is very complicated, we need to do each function block test to make sure what have we added or changed will not affect existing other function, this has to be done on every single product model we have made (Yes, we have much more hardware models today). Before each firmware is being released to public, it requires a so called ā€œregression testingā€, which is like to testing every single thing inside the software all over again. This regression testing used to take a week to finish, now it takes about three weeks.

  3. The software update is OTA, the user base is much larger today than in the past, so the update has to be planned very carefully then release to end customers to avoid any trouble. Some feature changes will affect more people than before and it requests more careful planning. Those changes that will result in sound change are much more sensitive. We always get customers saying they hear sound change after firmware update. We need to make sure most people (yes, we can not satisfy everybody) feel the change is positive. Since we have much more customers today, this is much more challenging than before, so we are very carefully about releasing new firmware.

Our current firmware development release way and schedule is more like Apple right now. We try to release what is called ā€˜big versionā€™ update every 12 to 18 months (like iOS 11, 12, 13). The big version change will bring major new features to the existing customers and this take long time to develop. The small version update (like iOS 13 to iOS 13.1) is mainly for bug fixes and minor feature changes and it will come or will not come depending on if it is needed, normally based on monthly or quarterly schedule. The last small number is usually a release that only fix a problem for existing release version without adding new feature. This is about to make a stable release version to perfection, for example from 6.1 to 6.1.1 to 6.1.2.

The software release history you have posted tells use the following story:

  • Version 6 is a very stable release because it only requires a few minor version bumps to get to a level that is stable enough for our team to move on to the next big version development (Version 7) . This is actually a very good release as many users are speaking high of it.

  • Version 5 was really a nightmare, it took a lot of updates to get to stable level (you have missed all 5.1 to 5.6, see this page for detail: https://support.auralic.com/hc/en-us/sections/205031948-ALTAIR-Device-Firmware). We did not release all those versions till we have 5.8 before we push it to large number of ARIES customers.

  • Version 4 was very stable as well, I can remember that our ARIES stays at 4.1 for a very long time as official firmware version before we bump everyone to 5.8.

I hope our 7.0 firmware will be a successful release and I wish it will, like 6.x, only requires minor updates to reach complete stability and customer satisfaction.

4 Likes

Thank you for taking the time to provide such a comprehensive answer.

ā€œCurrently we are trying to increase the intervals between software releaseā€

I think this is a statement that would be generally welcomed by many on this forum. I also think the explanation behind the burden of regression testing is also something that is helpful to be mindful of when it comes to the need to maintain an acceptable level of software quality.

On the other hand, it is also noted that it is the case that there are a number of threads on this forum raised by existing customers that have alluded to what appear to be ā€œcosmeticā€ issues rather than material feature changes. Maybe we can hope that there is a balance of addressing the UX asks in a timely manner, whilst still keeping the headline features on schedule. Appreciate that this is clearly not an easy challenge to solve with a small team and limited budget.

My last comment & a point of opinion, is that it surprises me how much emphasis reviews by respected commentators/publications place on hardware, where today I firmly believe that it is the software quality that makes or breaks this category of product. My own experience having had streaming hardware from multiple vendors + the fact people are prepared to pay $$$s for Roon has taught me to pay careful attention to this aspect. Also - take a look at the reviews of the client apps - so many express disappointment in this regard.

With this in mind I feel that a 12~18 month cadence might just not be enough when faced with more polished implementations in a market where there are some giants amongst niche players. Hopefully it is a challenge that Auralic can meet and am sure we all look forward to the 7.x release.

1 Like

Thank you. The core value of HiFi industry is still hardware. It is still the hardware (design) that mostly decides the sound quality about a piece of HiFi equipment. Software is about features and user experience.

I feel comfortable to add free new fun and free features to existing customers everything 12 to 18 months and I believe this intervals is sufficient. We will make small version update more frequently in the future to satisfy our customers need.

1 Like