I began the day yesterday updating to Java 7u51, after which absolutely none of my enterprise Java applications worked anymore. I could not reach the consoles of my Rackspace cloud servers. I could not open the iDRAC console on my Dell PowerEdge. They all exited with some error about the Permissions attribute not being set. Being the guy that I am I decided to search for the error. Turns out that 7u51 sneaks a major change in a point release: on the default Java security slider setting of “high” no applet may run if it’s self-signed, unsigned, or is missing the Permissions attribute.
Unfortunately, that describes all enterprise software, at least all the current versions of things I’m using.
This isn’t a trivial change. This is the sort of change that accompanies a major version, heralded far and wide for months, with customers given a choice about deployment and testing. Is that what happened? No, because this release is also a security update. So people across the globe autoupdate and suddenly can’t do anything, because absolutely no Java applets meet these criteria (probably not even Oracle’s own).
So into the Java control panel we go:
What sort of company labels the bottom part of a three-position slider “medium” when the description is “least secure?” Oh, a disengenuous one, that’s right.
The fix is basically to disable security, either globally by moving the slider (as I did, because I’m not a moron and can tell what the security prompt is for)[0] or for specific sites (like my entry for mycloud.rackspace.com). Of course, none of this is really what I want. I don’t want to trust mycloud.rackspace.com implicitly, because I don’t want just any applet running from there. I only want the console applet that I requested. I don’t want to lower all my security settings, either, but I’m going to, because I need to do my job.
Assuming that Oracle is trying to fix some legitimate problem, they’ve now completely bungled their shot at it. By changing defaults in what is essentially a point release they’re ensuring that no software has been updated to conform to their new standards, and users will have to change the security settings to simply continue doing their job. The right time and place for a change like this is a major version release, when all other parts of the support ecosystem already need to test and recertify against the new version.
Instead, it’s a mess, which is just par for the course when working with Oracle.
——-
[0] Pre-emptive snarky comment: “Well, that’s the problem they’re trying to fix, people are morons.” My coworkers and I have a saying, “you cannot fix people problems with technology.” This is squarely a people problem, and the “fix” here doesn’t make it less of a people problem because they botched it. Besides, if I’m an attacker I’ll just recompile my malicious applet with a Permissions manifest and go back to slurping up your credit card numbers. It wouldn’t surprise me to learn that malicious apps are already updated.
Sort of reminds me of Starbucks calling their smallest coffee “Tall”. Got bit by this setting with our HP iLOs and Netscaler GUIs. Java runs on 3 billion devices though, in case you didnt know.
In marketing I can see it — nobody wants something that’s small.
This is tech. Call it what it is. Actually, might as well just label that setting as “Go back to doing your work.”
In defense of Oracle — not that they are really deserving of it — I did see warnings about this when launching applets in a previous version of the Java 7 JRE. So at least in some cases users may have been given some advance warning that this was coming. Two of the dialogs contained a link to this page: http://docs.oracle.com/javase/tutorial/deployment/jar/secman.html
These screenshots are from JRE 7u45 when launching an IBM BladeCenter remote console:
https://dl.dropboxusercontent.com/u/8834653/java_security_warnings/security_warning_1.png
https://dl.dropboxusercontent.com/u/8834653/java_security_warnings/security_warning_2.png
https://dl.dropboxusercontent.com/u/8834653/java_security_warnings/security_warning_3.png
Pretty inconspicuous, if you ask me. I missed it, but I’ve been trained by Java (and everything else that throws an error) to just click through it all. Which is also a big problem.
Apparently they heralded far and wide for 5 months, though I agree it probably wasn’t recognized earlier by enough people.
https://blogs.oracle.com/java-platform-group/entry/new_security_requirements_for_rias
Hey man! It’s been a while. Hope all is well with you. 5 months is no time at all to an enterprise. Not long enough. And as much as I like to think blogs are the center of the universe that’s not enough publicity for it.
My main point is really just that this is the sort of thing that should be in a major version, not a point release. A better route would have been to add the new behavior so people could test, but leave the defaults alone so the stock JRE/JDK 7 behavior doesn’t change. Then in version 8 make the new default more secure.
It happened to me when I was installing and upgrading 10 IBM BladeCenter from remote…. :S
These Java changes are become so bad, that I’m now allowing the UCS Fabric Interconnect to work with simple HTTP traffic without the HTTPS redirect.
UCS # scope system
UCS/system # scope services
UCS/system/services # enable http
UCS/system/services # disable http-redirect
UCS/system/services* # commit
I used to fight so hard for security, to improve it, to lock things down. They’ve worn us down with incompatibilities and thoughtlessness to a point where we just don’t want to deal with it anymore. The risk isn’t really some hacker, it’s what will happen if we don’t get our jobs done. Or worse yet, if my job takes twice as long now and I miss part of my family’s life because of it. I don’t want to risk that.
When it comes right down to it, I am increasingly unwilling to waste my short time on this earth on thoughtless problems like these. Shut it all off and make life simple again so I can spend my time on what actually matters.
Had this same issue earlier this week with an EMC VNX admin panel. Used the same trick to get around it. While it is certainly not optimal, I will say I had been getting the warning about this Java app not working in some future release due to these permission bits for a while now. I would have expected a large dialog after the security update stating that “FYI – This is the release where we quit allowing these apps to run by default”.
Oracle is destroying Java …
And MySQL. And Xsigo. And Ksplice. And Solaris.
“you cannot fix people problems with technology.”
That lovely piece of sophistry is one you keep hearing in the tech industry and it is absolute nonsense at best. Fixing people problems is what we do with technology. It’s why it even exists. We solve people problems with it all the time, every day. It’s intrinsic in what we do as Sysadmins. It’s why we automate stuff. It’s why we put security permissions on things. It’s one reason why we have backups. The list is endless.
At least half the time that expression is said by someone, if you stop and really think about the context it’s more like “We don’t want to make the effort to fix this” or “we could fix it but it’s so inconvenient”, neither of which are good reasons and we know we’d never get away with actually saying. But no, we get that “can’t fix people problem” phrase as a lovely bit of sophistry over and over again.
The real problem is that *we* as IT Professionals have been accepting of Enterprise companies shipping unsigned software (I’m just as guilty of it). We expect software from our distribution’s repositories to be signed, and we tend to get antsy when required to install unsigned software on them, so why the heck do we not insist on it being done by vendors?
It’s not a complicated thing for them to do, we’ve just allowed them to be lazy, and we’ve opened ourselves up to some very interesting and potentially dangerous attack vectors along the way. Want to keep re-infecting a network once you’re in? Just replace the java control panel on $hardware with your own version of software. You can be sure the sysadmin will just blindly accept any dialogue box en route to running the software (which, by the way, has been telling users this change has been coming for a long time), and there you go the network is your oyster again.
Don’t kick back at Java/Oracle for doing the right thing, kick back at the vendors and the for doing the wrong thing, and the users (who really should know better) for tolerating it.
“Make it idiot-proof and someone will make a better idiot.” – Unknown
You’ve misunderstood the phrase. “People problems” not “people’s problems.”
Yes, we use tech to fix people’s problems. There are limits to how far we can go to fix people problems, like bad behavior, lack of interpersonal skills, etc. though. Sometimes a human just has to rub a couple of their brain cells together and think about something. What we’ve done is excuse people for not thinking about what they’re doing. Somehow, because tech is now our babysitter, it’s okay that a user clicked through all the warnings to install malware.
That’s crap. Your last sentence even says it. The users should know better. Hell, the users DO know better. But they do it anyhow.
As far as “we” goes in allowing unsigned stuff to propagate, try telling a CIO or CEO that we cannot go forward with an implementation because a vendor’s code is unsigned. That’s where the problem is, not with the techs. Fix that problem and you’ll fix the whole problem.
Oracle is not doing the right thing here, unless the right thing is moving liability from them to you. It isn’t their fault anymore for permitting an ecosystem of unsigned and insecure code, it’s now yours because you moved the slider and permitted the exception. Even if they have done something right, they’ve done it so poorly that it doesn’t matter in reality. It’s theater.
As we all know. The more secure it is. The less user friendly it is. and vice versa. Where do you draw the line? Some of my clients are automotive dealerships. 95% of their apps are web based, which require java and flash and Adobe reader. We install updates to keep the network and the data secure. And then the vendors do this. Yep it is definitely secure now! Better to not even bother with the updates. There was one user that was denied the ability to download files in Chrome due to Microsoft’s Windows Attachment Manager. First time that I heard about this thing. Google for a solution (maybe I should have used Bing), but I found nothing to allow the downloading of this file type (cvs). Users are what they are. But I am not like them. I don’t want nor need a program to tell me, that I can’t download a file. Or one to tell me that web app is unsafe. I can make that determination on my own. Sure end users are not so fortunate to have this ability. But they should at least be given the opportunity to make the choice. Not be completely denied access to a business doc. Completely insane!!! If the end user continually makes the same mistake, then fire them. Of course they need to informed, but after the 3rd time. No more chances!
The people problem is someone deciding for me, what is best for me. I don’t need that, nor do I want it. Why do these companies still use java and flash for their web apps? It is time to move on to something better. Neither are truly cross platform. Especially since the software vendors (oracle & adobe) don’t make decent Linux or BSD compatible software.
I think I’ve had the same problem. When you add the URLs to the exception list do the applications still run?
Well, we hit this during an terminal server update. A few thousand uses with error on their screens before their Monday coffee have kicked in… Board decision: Fuck JAVA. Let them BURN in hell! Off course its worded more politely, but when Oracle has turned into a wildcard that is run by idiots that tries to get other idiots to do the coding… Java have turned into such a train wreck in such a short time that everyone is dropping it. The price tag for dropping Java in our systems is way cheaper than trying to live with it.
And for those how claim that Java is still working: Talk to my hand. You know, that hand with the middle finger sticking up. 🙂
(And I’m so happy that I’m in a position where we actual can flip of Java and go “Nah, we dont need you”)