JCooney.NET

Like many people with a technical background I am deeply sceptical about the effectiveness of anti-virus software. Instead I’ve preferred to rely on an understanding of the risks involved with opening emails purporting to display dancing bunnies and the like, and relied on security principles like least privilege, regular patching etc.  In light of the problems McAffee users around the globe had a month or so ago, who wouldn’t question the costs and benefits of anti-virus? »

Since the release of the .NET framework there has been a quiet arms-race between those who want to protect their intellectual property, and those who want avail themselves of the intellectual property of others. And from where I’m standing, the reverse-engineers are winning, convincingly. 

The CLR is a virtual machine, and for JIT compilation, linking and introspection to work a considerable amount of type information must remain in an assembly when it is compiled.  Because of the extra meta-data (compared to a native C/C++/Delphi etc DLL) if you’re shipping an un-obfuscated assembly to your customers you may as well be giving them the source code. Sure they’ll miss out on the “why” in your comments, but other than that they get pretty much everything.  Tools like reflector just make it that easy to reverse-engineer un-obfuscated .NET code. This leads most people who are developing commercial products built on the .NET framework that are shipped to end-users to investigate obfuscation as a means of pr otecting their IP. Often this is also done to prevent easy circumvention of any licensing checks and restrictions that developers may have put into their .NET products.

Obfuscation can use a number of techniques, such as stripping meta-data, encrypting string resources, scrambling control-of-flow, adding rogue IL instructions that don’t break the program during execution but crash ildasm or reflector, aggressive method overloading (effectively another way to throw out information by removing method names), “packing” the application to hide the IL, and safeguards to prevent debuggers “attaching” as well as probably dozens more that I’m not aware of. And the reverse engineering crowd seem to have counter-measures for every one of these techniques.

Sometimes this takes the form of de-obfuscation tools specifically developed to un-do the work of a particular obfuscator ({smartassasin} for {smartassembly}, “the Xenocode solution” for Xenocode, DotFuckScator for dotfuscator etc). Usually this is a “partial” thing – after all the obfuscation process usually removes information which can’t be recovered, but the de-obfuscator will usually un-encrypt encrypted strings, allow the assembly to be viewed in reflector and un-scramble the control-of-flow obfuscation. No, I haven’t tried all these tools and some of them are now a little out of date, but I’m fairly sure similar ones exist for the newer obfuscation products, and it really doesn’t matter because the general-purpose de-obfuscation tools are even scarier.

And if the thought of obfuscator-product-specific tools isn’t worrying enough, or you were thinking about rolling up your sleeves and writing your own obfuscator have a look at this - a general-purpose re-assembler with de-obfuscation tools built in, demonstrated by Daniel Pistelli, author of the “NativeBlocks” application (currently unreleased, but you can be sure serious reverse-engineers have tools like this or more powerful). By running a few fairly short scripts through his tool he is able to remove a lot of the control-of-flow obfuscation in reflector (before Lutz Roeder gave it in to the care of red-gate/sold it). This is in addition to publicly available tools like Rebel.NET (a re-assembler for .NET, also by Daniel) and simple assembly explorer (a class browser and re-assembler, one of the few reverse-engineering tools I found that came with source…I guess if you know what you’re doing with RCE you don’t need source).

I guess it’s not that surprising that reversing of .NET applications is easy for those who know how. Even native applications that don’t have as many restrictions as .NET applications, and CAN quite happily redact all of their internal structure as they feel like during the compilation process are frequently reverse-engineered using tools like Ollydbg, IDA and Hex-Rays. Ultimately if it runs on a computer outside your control it can be broken. The ethics of all this reverse-engineering seems fairly dubious to me. Aside from the area of malware reverse-engineering I can’t think of many legitimate uses for this stuff, but that doesn’t stop me from being impressed by their ingenuity. 

Comments

That is the only logical conclusion I can draw from his recent comments. Either that or he is stupid. Otherwise why would he make ridiculous claims about Google’s recent collection of public WiFi data when doing streetview data collection, saying that “It is possible that this has been the largest privacy breach in history across Western democracies”. Sure…it’s possible. It’s also possible that the senator’s remarks are nothing more than a cheap use of parliamentary privilege to get back at Google for their speaking out about his ridiculous internet filtering plans.
Compared to the petabytes of search and adsense traffic, web crawl data, email and newsgroup postings that Google has been gathering for about a decade, this data set must be truly immense to be of such a privacy concern. Also his claim that the collection of data could not have been inadvertent shows a basic lack of understanding about how software works, and is built. Remember this (Google) is a company obsessed with data – A/B testing to determine which shade of blue in links converts better. So I’m not surprised that “get everything” was the default when someone there wrote some WiFi code for a different project, which was then re-purposed into StreetView. In spite of Eric Schmidt’s comments on privacy Google still has a pretty good track-record – remember Google vs. DoJ? Redaction of faces and care numberplates in StreetView?
Also the notion that Google need your permission to photograph your house is quite amusing. If I print the salacious details of my personal life on a billboard outside my house have Google violated my privacy because they know how to read?

I was looking at Word 2010 and WordPad (in windows 7) SxS today. There were a few things that occurred to me.

I often hear questions about .NET development and the Windows ecosystem that deal with very specific situations.  “What order do these events fire in?” “Will doing this leak memory?” “Will I see those details in a SQL trace?” “Will that be lazy-loaded?” When you’re a developer you often live and die by these kind of details, and humans are notoriously fallible – so why bother asking the question at all – why not write a simple program and find out?

I often see similar problems in the enterprise – questions that could be answered with a simple prototype, a cursory inspection of the data or the event logs, or a 5-minute discussion with the owner of an external system are left to fester and take on a life of their own. Meetings are called because we’ve got a problem calling system X’s web services. Really – what’s the problem? Well, we don’t know – we haven’t tried calling their web services. Sheesh.

Obviously it’s a judgement call on the part of the developers as to when to answer a question with a simple prototype and when to put up your hand and ask (and of course not all questions can be answered by working code). If the time it takes to ask the question properly (provide all the context, environmental details, sample data etc. to those you’re asking the question) is greater than the time it would take to answer it authoritatively yourself then DIY is a no-brainer. Similarly if you’re making an important decision that will be costly to reverse later then you’d really be foolish to take someone else’s word for it anyway, so you might as well answer the question with working code. Also, if you’re going to be blocked until you receive a response I usually ask the question, and then begin trying to answer it myself with working code (since I’m blocked anyway). Typically I find if you can’t answer the question fairly quickly with working code either you’re not clear about what you’re asking, or your question is too hypothetical (“will this work on MONO on Linux”? – if you don’t have a Linux VM lying around to test this hypothesis either there is a huge gap in your environment “test” matrix, or you’re asking a question that is purely academic).

A single data point doesn’t prove a hypothesis – all swans were believed to be white once, but all you have to do to disprove this theory is to find one black swan. If you’ve answered a question with working code, but are still doubtful go ahead and ask the question, with the evidence you’ve amassed from your code experiments included. People will see that you’re not being lazy and have taken the time to do basic investigation yourself, and hopefully people will only respond if they have something specific to add.

See also: how to ask a question http://catb.org/~esr/faqs/smart-questions.html

Comments

From time to time I see questions patterned like “why doesn’t Microsoft include X in Windows?” where X is some seemingly basic feature like PDF viewing, anti-virus etc. Surely Microsoft with its armies of programmers and war-chest of capital could buy or build a decent “X”? Well, here's the rub. Remember when Microsoft added a web-browser to Windows? The DoJ slapped an anti-trust suit on them, and the EU turned them into their personal ATM. »

»

I was listening to aninterview on Mixergy with Ryan Allis, an enterprising young man who built e-mail marketing company iContact into a company with >$3M in monthly sales, all by the tender age of 25. While this is a great accomplishment I couldn’t help but feel a little worried at some of the things Ryan was saying:We very happily, intentionally burn capital every month. And we do that purposely because of what is called, ‘unit economics’. »

The web is awash with stories of software developers who’ve “made it big“ - iPhone fart applications that allow the author to retire prematurely, companies raising millions in venture funding and being bought out by Google. I want my side project to become a huge success, but not for these crass comercial reasons per se, but rather the tangential benefits that financial independence would bring. The first reason is to avoid Bureaucracy. If I have to sit in another meeting and keep a straight face while hearing why someone should spend millions replacing a working $10K solution to “align with the organization’s strategic direction” I think I’m going to puke. When I saw the “standing on the shoulders of midgets” entry on the wiki I thought one of my friends was playing a prank, because it described the situation I’ve been in many times SO aptly. I had to read through the revision history and check the HOST file on my machine to be sure. The less I have to deal with things like this the happier I’ll be.

The second is to legitimize my being obnoxious. If you’re dirt poor and obnoxious then you’re just obnoxious. If you’re affluent and obnoxious then you’re “eccentric”, or “exacting” or “meticulous” or “detail oriented”. I want aura of righteousness that wealth brings to legitimize my bad behaviour.

Comments