To Upgrade or not? A look at Office 2016 for OSX

When a new suite of a familiar piece of software comes out, you have to decide if you want to upgrade or not.  Sometimes there's a long-awaited feature, and you can't wait to get the latest and greatest, and sometimes it looks an awful lot like what you've already got, so it doesn't really seem worth the bother.  The security and software risk profiles of the old vs new versions are another factor that should be part of this consideration.  

When we looked at scores for OSX applications, the Microsoft Office 2011 suite was at the bottom of its category, and the accompanying Microsoft AutoUpdate application was at the bottom of the whole OSX environment.  Since then we've been asked how Office 2016 stacks up in comparison, and it provides an excellent example of hidden benefits of an upgrade, as it has a much better risk profile than the 2011 suite.  

Where the 2011 suite was mostly 32 bit files, largely without ASLR, the 2016 had all 64 bit files, all with ASLR.  Stack guards are now consistently employed, and function fortification is used in 58% of the files, as compared to 5% before.  The main applications (e.g. Word.app, Excel.app, etc.) were at the bottom of the pack in the 2016 suite, meaning that these had not improved by much, but the overall score distributions, which take into account all of the binary applications and the libraries against which they are linked, improved greatly.  

The histogram shows the score distributions for a base install of OSX (El Capitan), with the 5th, 50th, and 95th percentile scores called out.  Each triangle corresponds to one of the scores called out on the hardening line below.  We're comparing the Average score, the score for the AutoUpdate binary, and the scores for the main applications (Word, Excel, and PowerPoint) between Office 2011 and Office 2016.  

The histogram shows the score distributions for a base install of OSX (El Capitan), with the 5th, 50th, and 95th percentile scores called out.  Each triangle corresponds to one of the scores called out on the hardening line below.  We're comparing the Average score, the score for the AutoUpdate binary, and the scores for the main applications (Word, Excel, and PowerPoint) between Office 2011 and Office 2016.  

2011 2016
Average Score 16.5 77
Minimum Score -5 54
Maximum Score 80 92.5

The Microsoft AutoUpdate file that was such an egregious offender also improved significantly, with a new score of 64.  It still has risky and ick functions (two calls to system(), which should make anyone in the security field a bit nervous about general hygiene here...), but now has stack guards, ASLR, partial fortification, good functions, and is 64 bit. 

The story wasn’t 100% positive, however – the Microsoft Error Reporting (MERP) application had a call to an ick function, system(), that hadn’t been present in the 2011 MERP files.  These sorts of bad and ick functions weren’t seen in Microsoft products for Windows, which implies that they might still be less strict about developer practices for OSX products.  Both suites had risky functions in most their files.  2016 had slightly more, but this could be because our risky function list was expanded to be more complete in its ANSI v POSIX coverage between the two data collections. [As a reminder: when we reveal our dynamic runtime analysis scores later this year we hope to share more information on our function lists and the math behind their weightings!]

Overall, the 2016 security stance was much improved, with much more consistent use of application armoring features, and much better overall static feature scores.  Perhaps most importantly, the AutoUpdate liability was significantly decreased.  This isn't a security feature that would be heavily advertised by the vendor, but it might be attractive to someone making the decision on whether to upgrade their corporate environment or not.  

Looking at changes in software packages in this way helps quantify and prioritize which upgrades may significantly decrease your security exposure, and which ones are more simply designed as feature enhancements aimed at selling you incremental revisions of the same tool. In the case of Office for OSX, removing Office 2011 and replacing it with Office 2016 is a measurable reduction in risk to your environment. However, we still do not see sandboxing, strict hygiene regarding bad and risky functions, and other advanced safety features in their OS X products (e.g. CFI and ShadowStacks) they way we do in their Windows 10 offerings.