HUMBUG logo

Planet HUMBUG











HUMBUGers

Feed: rss rdf opml

June 23, 2009

Jason Parker-BurlinghamWhere's that toad when you need it?


Borage and housefly
Originally uploaded by Nooks

Ugh, houseflies. I would have preferred a more interesting subject to explore the borage, but at least it did not fly away when the flash went off.


June 23, 2009 05:12 AM

June 22, 2009

Anthony TownsSide project #1: Pageant

So as per my post from a week ago, here comes the description of my first little side project. But first a quick reiteration of the aim: I’m trying to get a feel for what it’s like actually doing a tech startup; so not charging for my time, but rather making something once that I can then sell repeatedly without having to do a lot more work. This is intended to make me more experienced rather than wealthy, so “success” means learning something, rather than making much money. As a consequence I’m aiming for business ideas that are in the bad-to-mediocre range, that will nevertheless involve some interesting/useful technology. That way if the business part goes badly, I don’t feel like I’ve screwed up a chance to make a bazillion dollars, or wasted my time doing something pointless.

So the first interesting-tech/mediocre-business idea I have is related to popcon. I like to think a comment I made once helped inspire popcon’s existance back in the day:

I think It’d be interesting to have a debian-survey style package that when installed, informs the `project’ (stats@debian.org?)  who’s using which packages. This would allow us to get a *much* better indication on which packages’s are in fact moderately stable and tested, and which are just gathering dust; and give us a better idea of what’s appropriate for inclusion in stable and/or unreleased.

Sadly that mail disappeared from the web (it was in the archives mentioned at the bottom of one of my initial posts to debian-devel regarding (what became) the testing suite, but disappeared after an upgrade/reinstall of www.debian.org) — but it was nominally in the public domain as of late July 1998, and lo and behold, popularity-contest appeared some three months later, doing everything I’d thought of and more. (For all I know, my comment played absolutely no part in Avery’s implementation, but I still like to think it did :)

Anyway, cool as popcon (and my original idea!) is, there’re interesting ways you could extend it, getting more information, and doing more with it. You could, for instance, survey more information about packages — what version’s installed would give you hints about how many people are pulling from backports, or mixing stable and unstable, or Debian and Ubuntu; or checking conffiles against their original md5sum might give you useful information about how often the default configuration is sufficient. Or you could analyse the information more thoroughly — eg, seeing if there are any unexpected correlations between people who use particular combinations of packages, or doing a netflix-like “I see you use package foo, many other people who use it also use bar, maybe that might be worth investigating.” (I once tried to do that sort of analysis on the popcon data, but all I ended up with was a pretty animated gif, that apparently crashed some people’s browsers… Red dots were systems, blue dots packages, with a package being installed on a system implying attraction, and uninstalled applying repulsion)

You could also gather completely different data — like information about the hardware, or things like the default language or timezone, or potentially even things from logs. That would let you answer questions like “do many people run Debian on HP hardware?” or “which IBM hardware is popular with Linux users?” which might influence future hardware development or purchases; or tell you surprising things about where Linux is actually being used; or give you some feedback on questions like “is the OOM killer a common occurence?” or “is IPv6 adoption actually going anywhere?”

As well as just gathering data from otherwise passive users, you could also use the data collection as an opportunity to make introductions between users — having established you’re running Debian and have a particular Intel graphics card, you could be automatically given the address of a section of the Debian wiki that’s dedicated to issues with that card; with the idea being that you can see any helpful solutions other users have already come up with to problems you’re having, or leave your own tips for future users. The same principle potentially applies to other sorts of data: if you have an old version of wordpress installed, it might be reasonable to point you at some security alerts that apply to it, or having determined you’re running Debian on some HP server, you might get directed at some updated management software that enables some extra features.

Another interesting improvement I think you could make is to provide ways users can aggregate and anonymise their own data. Even in the age of social networks and ubiquitous transparency, managing privacy of this sort of data is important: it would be spectacularly bad to provide a website that told people exactly which machines were vulnerable to which secuirty exploit, but that’s exactly what a list of which machines have which versions of which packages installed would provide. The popularity-contest software goes to some lengths to avoid that, by identifying data against a randomly generated UUID rather than an internet address, email or username; by not storing detailed information about package versions; and by restricting who has the ability to run any detailed analysis on the data. But you can go further than that by aggregating and filtering the data even before it makes its way to a centralised server — eg, rather than have each individual machine on a network reports its statistics to Debian, you could have the information sent to a proxy server that aggregates all the packages into a single report (30 computers, 10 of which have apache, 15 of which have exim, …), thus removing certain correlations (do all the machines running apache also run exim? or do none of them?), and potentially filtering things like the UUID (which might reveal something about the random number generator, particularly given Debian’s recent issue with randomness…) popcon version (which gives an indication what version of Debian is in use, and in some cases how recently it’s been updated) or timestamp (that may give away that the machine has been down). And if you’re running a network that’s intended to be somewhat locked down, it might be more reasonable to have computers reporting to a machine that you control, rather than one just out their in the wild.

So that, in very rough terms, is the spec for this project, which is currently going by the name “pageant” (ie, a popularity contest that takes itself a bit more seriously…) The technical goal is to provide a pageant client that people can run on their systems, which can report potentially arbitrary information to a central server and can receive and present relevant snippets of advice related to that information; a pageant proxy that can intermediate and filter pageant clients to provide a slighter higher level of anonymity/privacy; and a pageant server that can collect the data, provide relevant advice to clients, and analyse the data. I think it’s feasible to do an interesting job of that, that should go a little further than existing programs, and be usable by actual people, though I suspect the server side will have to be a bit beta-ish to be finished within a  week or so.

The business goal, obviously, is to turn some of the hypothetical benefits touched on above into actual income, ideally without turning it into a vast NSA-like data hoarding corporate conspiracy. I figure there’s a few reasonable ways to approach that:

Anyway, that’s the project! My notes have a few other things in them worth mentioning — there’s a couple of not entirely little complications in a few of the above ideas, for one — but this is already long enough, and it’s not like I can’t blog again later. Even though there’s a few similar projects around (popcon and smolt in particular) I’m planning on taking a NIH approach and starting from scratch, on the basis that current stuff is mostly pretty basic to reimplement, and getting an architecture I’m comfortable with is pretty important in making it appropriately generic. As always, helpful tips, questions and/or any general encouragement appreciated, either by email or the comment link…

Permalink | No comments

June 22, 2009 07:45 PM

Jason Parker-BurlinghamAnn has the setup shot for this


Male goldfinch at the feeder
Originally uploaded by Nooks

Nothing more complex than the macro lens on a tripod with a remote shutter release. Later I got a little more fancy and used a focus trap, but this is manually focused using a small twig as a guide. The camera was in shutter priority to try to freeze motion but this was a mistake since the shutter speed stays constant by varying aperture and affecting focus. Later pictures with lower ISO, slower shutter and wider aperture worked just as well, but had less compliant subjects.

I could have gotten closer but that makes focusing even more chancy (there are lots of pictures of very blurry goldfinches in my collection now) and the birds seemed to not appreciate a camera closer than a few feet away.

Lessons learned: go full manual next time, don't bother with the focus trap, I think it just made trying for a shot more difficult. An actual cable release instead of an IR remote makes taking a shot less chancy. An off-camera flash may be useful. A sandbag and string to keep the feeder still could be useful too.

Interestingly only a few male finches ever came anywhere near the setup: a shame since they're a vastly more colorful bird.


June 22, 2009 12:35 PM

Adrian SuttonI Love Parser Generators, I Hate Parser Generators

Parsers drive me mad.I was reminded on the weekend of how much I like working with parser generators – they’re just so pure and clean. You really feel like you’re working with a grammar and all those CS lectures come flooding back. Writing code to parse the same content by hand just never has that feel. Plus they create incredibly accurate parsers in very little time at all.

I was also reminded of how much I hate parser generators. They generate very accurate code which is great when you have very accurate input. In the real world, it just means the parser craps out an awful lot on very minor syntax problems. So then you try to make the grammar more flexible to accept that input and the generator just complains that the language is no longer LL(1).

Off we go into the deep dark depths of those CS lectures. Now all of a sudden you find yourself with pen and paper out drawing states and the paths between them. Pretty soon you want to migrate to A3 paper and then on to butchers paper. Eventually you find yourself writing on the wall.

Real world content just isn’t sane. You can’t tokenize it first and then just use those tokens – characters in different places have all kinds of different meanings. You don’t really want to validate the content as you read it1, you just want to do the most brain dead simple thing to get that content in and in a form that you can work with2.

I run into this every time I work with parser generators and wind up spending so much time making the grammar fully tolerant that it winds up being easier to just write the entire thing by hand. I just can’t help but think that there should be a better way though.

1 – beyond making sure you’re avoiding buffer overflows and that the resulting model isn’t dangerous etc but often those kind of checks are best done in the code that actually does the work (i.e. assume every value is user supplied rather than assuming that the content is all nice and safe).

2 – this of course is situation dependent. I happened to be parsing CSS where tolerance is the key to success. Parsing configuration files on the other hand should be strict and fail fast.

June 22, 2009 07:52 AM

June 21, 2009

Ben FowlerDaily Mail features JET and MAST!

'This is save-the-world science. This is the energy holy grail'This Daily Mail article is pretty awesome, with some great photos (far better than mine), and the cool JET video. If we want fusion to be taken seriously as a potential energy source, we need more of this!

June 21, 2009 10:32 PM

Ben FowlerVisit to UKAEA Culham: my photos

I've posted all the pictures I took on my Culham trip, up on my SmugMug page.Check them out here.

June 21, 2009 06:36 PM

June 18, 2009

Ben FowlerVisit to UKAEA Culham: is public outreach a waste of time?

When I was 17 and working as a printer in Quickfast Photos in Surfers Paradise, I had the opportunity to catch a bit of the IndyCarnival. My boss, Andy, besides being a raging pederast, was extremely cheap and wouldn't buy tickets the legal way, so he got transit passes usually reserved for the locals, and spent the entire carnival watching the race while I held the fort at the shop. However, I

June 18, 2009 08:03 PM

Ben FowlerVisit to UKAEA Culham

Yesterday, I realized an ambition I've been harbouring for quite a few years -- to visit the world's biggest tokamak fusion reactor experiment at Culham, near Oxford.When I was a kid, I remember marvelling at pictures of the Joint European Torus, and since then, I've closely followed the construction and commissioning of the UK's own fusion experiment, MAST. Mankind's future will be made or

June 18, 2009 07:37 PM

Sarah SmithThe Answer Man

Photo by Elliot Moore http://www.flickr.com/photos/elliotmoore/
How do we save the world as it teeters on the brink of an energy crisis and a climate change catastrophe?

Many governments now are scrambling to put together a coherent set of answers: some sort of plan to address this question.

I recently read a slim volume by investigative journalist and author Edwin Black which steps up to provide such answers.

"The Plan" is nothing less than a no-holds-barred manifesto of energy security politics. It reads as a apocalypse survival plan, detailing week by week how western civilization might save themselves when the black gold stops flowing, and anarchy rules our streets.

From my perspective - not being a politician looking for a policy - the most eye-opening part of the book was the coverage of the fragility of oil supply. The book opens with this important background, before it gets into the survivalist stuff.

Oil has over a century ago stopped being a commodity - it no more follows supply and demand or any other market place rules than nuclear weaponry. Oil is a political issue.

Enter the IEA. This is an organization dedicated to protecting nations who join it from the impact of oil supply loss. Set up after the oil shocks of the 70's, the IEA member nations recognize that loss of even a fraction of supply would lead to an apocalypse - the signatories agree to help each other in the event of such a calamity by immediately executing a three-pronged action:
This material is compelling reading. It puts a lot of what I already knew about the Oil Shocks of the 1970's and later into chilling perspective.

But doesn't this sound like a plan?

It would if the first two items would address a tiny part of the impact of an oil shock. Even a drop in oil supply to the USA of 5% would trigger the stock draw - raiding the Strategic Petroleum Reserve. The month or so that the SPR would last is nowhere near enough time to enable demand restraint. Anything that had not been already prepared would be overwhelmed by chaos before it got off the drawing boards.

And surge production - the USA is already using and drying up its own oil wells. If the 5% drop is more widespread than just the USA then the chances of other countries being willing or able to surge supply enough to get the USA out of the fat-fire are near zero.

But the big flaw is demand restraint.

Demand restraint? Are you kidding?

If there's one thing that will get an American president shaking in his boots, its trying to get US citizens to restrain their gas guzzling ways. And a century of GM marketing teaching them that success is the same as a big car is going to take a long time to turn around.

New Zealand responded with great restraint during the Oil Shock times by instituting carless days.

USA has no plan and no ability to do anything like carless days, even though its obligations through its membership of the IEA decrees that it should.

But do we really have to worry about another oil shock?

Black shows why we should expect it as a virtual inevitability. He covers a range of threats from military, through economic, to pragmatic which will likely realize the substantial ruination of oil supply.

It won't take much.

After Hurricane Katrina hit the continental USA and took a number of refineries offline, the availability there dropped by 5% - this is the figure sufficient to trigger releases from the Strategic Petroleum Reserve.

A drop of 7% is enough to precipitate an international crisis, and trigger obligations under the IEA membership.

Losing 10% of oil supply in the USA is unthinkable: so "off the charts that we can't model it" according to one of Black's sources.

This is because the USA has already hit its Peak Oil - its been many decades since those cocky Texas oilmen were supplying anything more than a trickle of the black gold gushing into the USA's unquenchable gullet.

Around 60% of the oil used in the USA is imported. But what is more interesting is that due to Oil Market intricacies, while it would appear that most of that oil comes from friendly countries such as Canada - the true picture is that USA is far more exposed to the middle-eastern oil markets than raw import source percentages would suggest.

But even looking at raw figures, Saudi Arabia supplies over 10% of US oil.

This fact alone leaves the super-power critically vulnerable. Its why twice during his presidency as Black describes, George Bush went cap in hand to the Saudis asking for price breaks. The volatile nations of the middle-east have the the international oil market in the palm of their hand.

The remainder of The Plan presents a range of energy initiatives that would allow a shattered country to try to survive the impact of an oil apocalypse:
Of course Blacks point is that if the Plan is instituted now, instead of waiting for the disaster then maybe we can pull back from the precipice.

Hydrogen Update

This is an update on an earlier blog article about Edwin Blacks book "Internal Combustion: How Corporations and Governments Addicted the World to Oil and Derailed the Alternatives".

Mr Black responded to my offer of commentary - his comments are in red below.

Where as for hydrogen:
A 2008 report from the National Research Council estimated it would take $200 million from government and industry over the next 15 years to commercialize hydrogen fuel cell vehicles to the point they could be competitive with gas vehicles.(Reported by reuters)

These are just politicized numbers. Last year, I drove a GM hydrogen car that used water dispensed from a Shell station to create the fuel.

The problems of practical transport using hydrogen are well known - see this article in Popular Mechanics for an even handed analysis.

Hydrogen has its hands as dirty as oil, since it is traditionally made from natural gas, a fossil fuel. Companies mentioned by Black in his chapter on hydrogen, such as Linde, and Air Liquide are now leading members of the National Hydrogen Association, along with guess who: General Motors and Shell.

A common feedstock is not natural gas but pure water.

There are 3 main ways to make hydrogen--reform it from natural gas as some wish so they can use existing infrasture, electrolyze it from water like the Shell station in Santa Monica ( I favor), or use bacterial generation which still being perfected to yield higher amounts.

Hydrogen could be synthesized from your hat--it is in 99% of the universe.

Hydrogen futurists such as Dr Alan Lloyd were pivotal in killing off the California zero emissions mandate. The ZEV laws brought in by the California Air Resources Board to fight chronic smog had led to a huge flurry of electric vehicle activity in California. In the period where Lloyd was the Chairman of the California Air Resources Board and Secretary of the California EPA the electric vehicle innovation and recharge point infrastructure rollout was crushed literally, and replaced with a highly touted hydrogen program.

I know Lloyd. He was not a hydrogen futurist. The hydrogen fuel cell was invented in 1838.

Lloyd was also Chairman of the California Fuel Cell partnership. He vehemently denied that this was a conflict of interest.

Of course proponents of hydrogen with vested interests ranging from pet research to major investment, are ready with all sorts of arguments against the view that electric is superior to hydrogen for transport. Even if they're not all Lloyds with a cash register tune playing in their ears, their views are based on theory.

Remember Sarah--the hydrogen car is just an electric car with the electricity generated from as hydrogen reaction. Now you can generate electricity from a battery, from on-board hydrogen, or a squirrel in a cage--just the end product is all the same--electricity out of the fuel combination to run the wheels. Batteries are a dirty business. It is brown green, not green green. Water is clean. It only takes a zap of electricity to electrolyze it. In Vegas that is done with solar.

Electric Vehicle Update

In October the Australian Electric Vehicle Association has its first ever field day. Yours truly will be there and I plan to report on the event.

The guys from the AEVA have been building and driving electric vehicles, approved for driving on our roads now for decades.

I agree with Mr Black that hydrogen can be generated cleanly, and I hope some day soon that it will.

I believe that avoiding the Oil Apocalypse will take a broad ranging response - a response from all of us. And an effective robust response will involve a range of technologies.

We have to start now - and we can start with non-oil burning technologies that we have on our roads right now - and for me electric vehicles are the most exciting reality, but of course CNG and LPG will play a role, as will biodiesel.

I'm sure that the current success and innovation in electric vehicles will pave the way for a viable hydrogen powered car, since as Mr Black says, hydrogen is essentially another way to power an electric vehicle.

I look forward to seeing a hydrogen vehicle on our shores as soon as possible.

In the meantime, come and join me at the AEVA's electric vehicle open day!

June 18, 2009 11:25 AM

June 17, 2009

Jason Parker-BurlinghamSummer's first cookout


Sparks flying
Originally uploaded by Nooks

On Saturday night we were having trouble deciding where to eat and settled on buying hotdogs and the makings of smores and lit a small fire in the driveway. I got to light the fire and Ann was exceedingly patient while I did (we both like lighting fires, what's not to like?). After a time I decided it might be fun to have some pictures of the flames and Ann suggested a long shutter speed would be good. Henry helpfully ran a long stick over the length of a burning log to throw up some sparks.

Sunset at nearly 9pm: check. Hot dogs: check! 70—80F heat: check! I do so enjoy summer—it's part of the reason I adored the Bay Area, where summer is a season 6 months long.


June 17, 2009 10:33 PM

Andrae MuysLooking for JSON, ReST, (and in-memory RDF) frameworks

Currently writing a number of small web-services to do various informatics tasks (more detailed post to come). Fortunately I'm not the one having to deal with 3rd-party SOAP apis! Still I do need to do various XML and JSON parsing, and not having addressed the latter before I've gone looking for libraries.

Currently I am about to start using Jackson, but was wondering if anyone had any warnings, advice, or recommended alternatives? In the course of looking at what was out there I have also come across Restlet, a ReST framework that seems like it is well worth the time to figure out and deploy, so I will probably be doing that soon as well, any warnings or advice on this will be welcome.

One of the nice things about Restlet is its support for RDF. Granted it doesn't support querying, and the terminology in the interface is a bit confused, but it does use its native Resource interface for URIRefs, so it should integrate well. OTOH, if it does prove useful as a ReST framework, I can see myself writing a quick Sesame or Mulgara extension, as there is only so much you can do with RDF before you need a query and/or data-binding interface.

June 17, 2009 04:37 PM

David StarkoffThe inventive step

When Justice French, then of the Federal Court of Australia, was nominated to be Chief Justice of Australia, the quality of his extra-judicial speeches was noted. It is a rare judicial officer who can segue from declarations to Homer Simpson, or use Judge Dredd to illustrate arguments about judicial activism.

As a highlight from his recent speeches, I thought a part of his 4 April speech to the Licensing Executives Society of Australia and New Zealand bore special mention.

Despite these adventurous extensions of intellectual property law most of the words it uses remain boring. … Terms such as “inventive step”, which sounds like a dance created by a nerd … are not terms to stand your hairs on end.

June 17, 2009 09:49 AM


Last updated: June 24, 2009 01:37 AM. Contact Brad with problems.