The Five Software Development Tools You Wish You Had

Last week I went to the embedded systems conference (ESC) in San Josa, CA. This is my second time going and last time I was unimpressed. This time however there were five technologies which really shook my cookie.

All of the products come from different vendors so there is no chance that we will get all of them in a single bundle any time soon. Also, they all cost big $$$ so don't expect them to be GPL'd either.

5) WindRiver's Sensor Points - This is a pretty cool technology which is like an automatic update API for arbitrary code. It allows you to send and run arbitrary code to any system running VxWorks. Thus if you find a bug after your product has been released, you can send a patch to the system while its still running. There are some security concerns here which I never got an answer to, but I'm going to assume (hope) that WindRiver has got those worked out.

4) Virtutech Hardware Simulators - These are some pretty insane simulators. Once you define your hardware model they run the hardware on your host as a Virtual Machine. In many cases it can run even faster than real-time. But thats not all, you can turn on checkpointing in the simulator and record the state of the simulator over time. They you can actually run the simulator backwards and forwards, and even send the checkpoints over the internet so others can see view its state as well.
Sounds great right? The catch is creating the hardware model. Making new models is both technically challenging and can take months. They have a number of models already created but if you have your own funky hardware you're probably going to have to do a lot from scratch. Virtuatech makes their money by creating the model for you... which can be expensive.

3) QNX's Multi-Core Analysis Tool
Multi-core is taking off and managing all of those cores is not going to be trivial. I took a look at this tool and I was pretty impressed. You can see what each of your cores is doing at any given moment. It tracks which programs are running and what types of system calls and interrupts are occurring on each core in a nice easy to use graphical interface.

2) Coverity Static Analysis (and others)
This product has got enough of its own publicity so I'm not even going to go further here. However I will add that it is really cool, and everyone should be using some sort of static analysis (if they can afford it).

1) Green Hills Software and Lauterbach's Back-in-Time Debuggers
These technologies are really cool... you have to see em to believe em. They record the state of your system at every point in time and lets you go back in time and check out memory, registers and more. This would be helpful for tracking down hard bugs and extremely useful just in general.

The catch is that you need special hardware to run this stuff though green hills has some workaround technology called TraceEdge.

WindRiver and RTLinux .... My take on it

I know the news is about two months old, but I don't like to rush in to commentary...

WindRiver's purchase of RTLinux is the first smart move that WindRiver has made in years. Before their new acquisition, WindRiver only sold their legacy OS VxWorks and a six-month old version of linux kernel they downloaded from kernel.org.

Let me just rip into VxWorks for a bit, cause I have some well needed venting to do.....

Technologically, VxWorks lags behind every other major embedded operating system. Its amazing that the most popular OS in the industry is also the worst. One of their more recent innovations, memory protected processes, has been in most embedded operating systems for years.

Also their ARINC-653 partitioning is pathetic. They implement ARINC by running a master VxWorks OS which schedules slave VxWorks OS's which each runs its own ARINC partition. So if you want three ARINC partitions, you will need to run four copies of VxWorks on your hardware. Pretty much all other competitors can sanely implement ARINC with just one OS.

The main argument for buying VxWorks, is that your code is locked into it and you have no other choice.

... End anti VxWorks Rant

So even though VxWorks continues to make WindRiver a lot of dough, its not going to get them anywhere interesting.

On the other hand, their purchase of RTLinux is interesting. They have a real chance of bringing enterprise software developers into the embedded world. Most embedded products only have a small amount of hardware-related or realtime code. Everything else is pretty similar to desktop development. If RTLinux can get every-day software engineers to work on an embedded systems, it should become pretty popular.

Buying RTLinux brings WindRiver's technology to the forefront and I hope it moves our industry forward. However I doubt Windriver will make any money off of it. Whenever you sell Linux products, you also have to give them away for free. Why would anyone buy RTLinux when they can download it for free?

.NET Microframework

So Michael Hall and the rest of the .NET nerds are making a big rucus over their .net microframework. They promise to let all of the C# junkies start writing code for the next generation ipods and cell phones. Most C# junkies don't know what malloc is (let alone an interrupt) so I am skeptical. But if Microsoft can get non-embedded software engineers into embedded technology they will make the big bucks.

The problem with the microframework is that it does not have any real-time support and only runs on ARM. There are very few BSPs (I think only one!). This API is a good start but what is really needed will be more of a hybrid.

What you really want is to write all of your real-time code on a true embedded OS, and the high-level non-critical apps in a managed language like C# or Java. Microsoft has not pulled out a solution like that yet, but others have.

AONIX has a real-time java VM which runs on top an embedded OS. It doesn't look like its catching on yet, but I do believe that the technology is the way of the future (give it five years).

Until Microsoft comes out with a solution that plays nice with real-time systems, all they are offering is an API with a huge memory footprint (half a Meg!), horrible threading support, no partitioning, and crappy performance. Good work Microsoft nerds.

Ripping Apart Microsoft Visual C++: Performance

I said I'd do it and I never lie. Lets tear apart Visual Studio. For now I'll start with performance, not my favorite topic, but I'll get to the debugging features later. Each one of these entries deserves its own bog post, but who cares, you probably just think this is flame anyway (I think its thoughtful analysis!).

Performance
1) Starting Up (and Stability)
2) Loading DLL debug modules
3) Running a program

Performance
1) Starting Up
Normally I don't really care about program start up time. I use Visual Studio extensivly, once I start it up, I should never to start it up again. For me its a constant-order performance hit which should only effect me when I restart my computer. Most people restart the computer once a day, I do it much less. Waiting an extra ten seconds a day isn't a big deal.
However Visual Studio crashes. It crashes a lot. I find that halting in the wrong place, or stopping debugging at a key moment will cause it to crash. This happens about 4 to 5 times a day. Therefore 4 to 5 times a day I find myself restarting Visual Studio, loading my project, and pulling everything back into RAM. Altogether its probably a 45 second ordeal which is over three minutes a day. Not a biggie, but definitely an annoyance.

2) Loading DLL debug information
Why does visual studio have to load all of the debug information about a DLL the instant its loaded into program memory? It doesn't take long to load each DLL, probably about a quarter second. However most modern win32 apps link in about 5 to 15 DLLs (complicated programs much more). Most of the DLLs are loaded up at the beginning of program execution so there is a visible delay waiting for them when your program starts. That means I wait two to three seconds every time I want to run a program, even if its a simple console app. If I run the console app alone it completes in a fraction of a second.

This is only a real problem with the CLR. I'm guessing that its a problem with JIT caching or something, but I'm not too sure.

3) Program Performance
A good debugger should be as passive as possible. However running a program within visual studio is significantly slower then running it standalone. I'm not too sure why but I'm guessing that my item above plays a large roal. It could also be that Visual Studio installs all sorts of exception handlers and instrumentation to get things working. With managed CLR Visual Studio may be doing much much more. But its all really speculation.

Blogging on weekends

Why is it that there are never any good blog posts on weekends? Its almost as if people are not in front of computers or god-forbid.... outdoors.

Getting a new boss

My boss is quitting my company. He wants to move back with family and have kids and stuff. Good for him, but crappy for me. The guy who is supposed to replace has some very big shoes to fill.

My old boss was excellent. He cared about and believed in the product we're making, he supported my colleagues and I to the best of his ability. He let me have very flexible work hours and was easy about taking vacation days. Most importantly he was extremely enthusiastic about our work and it was contagious.

The guy who is going to take his place isn't bad. I know him and he's a nice guy. But he isn't in the same league as my former boss. Additionally he is about to start managing a huge code-base that he knows nothing about. My former boss wrote/architected a good amount of our code and now this other guy is supposed to come in and take it over.

It'll be interesting to see how it plays out.

Crash the Visual Studio Compiler....

Wanna know a real easy way to crash the visual studio compiler? Compile the following c++ file using Microsoft's CLR:

_w64 a;

For some reason the Visual Studio compiler really doesn't like _w64. Its pretty funny.