In our previous post about the score histograms for Windows, Linux, and OSX, we promised deeper dives to come. We also noted interesting things about each continuum and reminded people that the real value is being able to compare risk present in various software within a single continuum.  No we will take our first look at where some applications of interest live on the score continuum for OSX.  We'll look at three categories of software here: browsers, office suites, and update software.  All of these are targets for attack, so security is a concern for all of them.  The extent to which security influences purchasing decisions in these categories likely varies a great deal, however.  We'll discuss this aspect in more detail for each group as they're introduced. [Note: As stated before, these articles are to familiarize people with the value and limitation of static measures of risk. Once we finish these blog posts we will begin describing our dynamic and predictive analysis values and how they work with our static measurements.]

First up are browsers.  Browsers are particularly interesting because it's easy to change which one you're using, and easier for security to be a main consideration in the choice of which to use.  After all, they all provide you access to the same web content, they're a major vector for infection, and most of them are free, so cost isn't usually a deciding factor.  For corporations it is often most efficient to pick a single browser to configure and support. So if you are an OS X based shop, how would you chose the browser with the strongest security build and developer hygiene characteristics?

You might remember that the histogram of all OSX El Captain scores looked like so (as of August 2016): 

The scores increase as you go from left to right, so applications lacking basic security artifacts will fall further to the left, and better built applications (from a security/risk perspective), will be further to the right.  To make this more concrete, let's look at where our three main OSX browsers fall on the continuum.  

The line below the histogram is our relative hardening line. It is mapped to the histogram above, and used to notate where a particular piece of software (and it's supporting libraries) falls on the soft-to-hard scale. The yellow triangle on the histogram above the relative hardening line is aligned with its binary package below to help the viewer see where they land in the overall distribution.

With the browsers, the story is pretty clear - there are pretty big score differences between our three exemplars currently available on OS X, with Firefox (v 39.0) close to the 5th percentile mark, Safari (v 9.0) in the middle (of the browsers, but below the 50th percentile mark for the OS X environment), and Chrome (v 48.0.2564) leading the pack with a more respectable score.   We were surprised to see that Firefox had no ASLR enabled, so we did some digging and discovered that the feature had been intentionally disabled by the development team so that it could be backwards compatible with OSX 10.6.  That backwards compatibility is planned on being dropped in the next major Firefox release, v 49. 

Keep in mind that the scores for these browsers and other applications, are only meaningfully compared within the same operating environment. For Windows 10 and Linux the same browsers we are showing here in OS X ranked and scored significantly differently. No wonder there has not been a clear winner to the browser security holy wars (yet); depending on the browser and the OS, the correct answer differs! When we measure these features, it becomes readily apparent as to which browser is more difficult for an adversary to newly exploit.  Perhaps one of the browser developers will use the information we will be producing to become the clear leader across all available platforms.

This leads us to one of the questions we get asked a lot, which is how often will we reevaluate products?  Will we update our reports each time a patch is issued, for example?  We could do this, and in fact we have during our development process, but it is not our current plan. Extracting the hundreds of build features, complexity measurements, and artifacts of developer hygiene for the static portion of our risk analysis only takes, on average, a few seconds per target and it is easily distributed and performed in parallel. However, it turns out that in practice we really do not need to re-evaluate for every minor revision, because a single minor patch usually doesn't change enough to matter.  For organizations and consumers who provide funding for such services we could do this, and for the fine grained feature analysis that we described in the BlackHat and Defcon talks this can be meaningful, but in general we plan on focusing our public reports on each major release of a product. Major revisions have shown significant changes for better and worse in the products we have analyzed, but minor changes from patch to patch seldom move the needle much.  As an example, looking at several versions of Chrome [48.0.2564 to 50.0.2657] showed precisely the same overall score.  After all, a patch typically only touches a small fraction of an application's total lines of code.  The purpose of the patch is to fix a specific bug or vulnerability instance, not address whole classes of vulnerabilities or change fundamental design elements (unfortunately).  

Our next category of reviewed software is office suites.  Office suites (typically word processing, spreadsheet, and presentation software), are a significant target in phishing and spearphishing attacks. If a user opens an attachment in an office suite application it is largely up to that application to prevent malicious activity and execution. While traditionally there has always been one dominant suite that everyone uses by default, there are options available here.  On an environment like OS X there are even multiple commercial-quality options.  We do appreciate that people have a more difficult time switching between suites, though, because office suites vary more in their features and capabilities, so security usually is not a sole driving factor in decision-making.  The average consumer might not be aware, however, of the options that are now available to them, or how much the security might vary from one offering to another within OS X solutions.  In the following chart we overlay Microsoft Office 2011, iWork, and open source office offerings (LibreOffice 5.1.0 and Apache's OpenOffice 4.1.2) on the existing continuum that we just populated with the browsers above.

Each office suite's software is based on the average of the scores for their main applications - their text editor, spreadsheet tool, and presentation application. For each of these applications the score is made from the core application, all of the libraries it loads, the libraries they load, and so on, as evaluated for the features described in our prior post. So, the average of Word, Excel, and Powerpoint in Microsoft's office suite, or Pages, Numbers, and Keynote for the Apple office suite.  

Take a moment to reconsider the previous blog post where we showed the histogram scores for Windows 10, OS X, and Linux. The histogram of scores for the base Windows 10 install showed that Microsoft was putting in a fair amount of effort to consistently use application armoring and good developer hygiene.  It is interesting, and disconcerting, to see how different that picture looks when you see Microsoft's offerings as built for a competitors' environment.  Microsoft's Office for OSX suite is all 32 bit binaries with executable heaps and no source fortification.  The vast majority of OSX 32 bit binaries had non-executable heaps, which means that the build environment for this office suite is either very out of date and/or idiosyncratic, or that this particular safety feature was specifically disabled.  Either explanation doesn't reflect very well on Microsoft.  

The other office suites do better, all landing around where Safari was.  We looked at both OpenOffice and LibreOffice, but they had enough code, build, and hygiene features in common that they are essentially interchangeable from a perspective of static binary scores, so they only have one data point on our chart.  The native Apple office suite does the best, although not by a huge margin.  These tools at least applied the default application armoring features that most applications had, and were 64 bit binaries, which is why they end up in the middle of the overall distribution.  These are all relatively soft targets, though it still makes sense to choose the one that introduces more work to the adversary from a security and safety perspective. It is unsurprising that new zero days continue to target these applications in this environment.

Finally, we looked at Auto Update software.  This isn't a category of software that consumers specifically choose to run, but rather something that they get as a side effect of other purchasing decisions.  Still, Auto Updaters are a major security target, and their security contributes to the overall cost of ownership for the products they're a part of.  After all, they accept some form of input from remote sources and they are often in a position of privilege within the operating system for installs and upgrades. The following chart adds Microsoft and Apple's update software to the continuum. One of the results is... disturbing. 

We always look across all of the binaries to identify the very best and very worst applications in any run, to see if anything of interest pops out.  We were dismayed to see that Microsoft AutoUpdate (the update software that ships with their office suite) had one of the worst 10 scores out of all 38k+ binaries we evaluated.  It's a 32 bit application with no ASLR, an executable heap, unfortified source, and no stack guards.  In the office suite scores from the prior histogram, we did not include this update software. If we change the score of the office suite to include all the software it comes with, this pulls the score down significantly. It should be pointed out that this updater is related to Microsoft's DRM (digital rights management - anti software piracy) component that is listening on the network on your OS X system if you have MS Office installed. 

In contrast, the native OSX Auto Update software does quite well, scoring slightly better than Google Chrome did.  Given what a target auto updaters are for attackers, this is more along the lines of what we would like to see for this sort of software.  

It is clear that there is a variety of products available to consumers, with a lot of variation in their security and design quality.  And, as we see with the Microsoft products for OSX, a vendor that does well in one environment might do quite poorly in another.  There is no guarantee of consistency of development practices across a large organization.  However, with information such as this we are hoping to arm and inform consumers to help them make better and safer choices for their particular environments.