csimms.botonomy.com

Easy Way to Test Offline HTML5 Web Apps

Assuming you've already written your cache.manifest and everything according to the HTML5 Offline Web Applications spec, you'll want to test it. So far, the easiest way I've found to test is in the chrome browser on your desktop. To test:

1. Go to your url. Keep your javascript console open, and you should initially see messages about the "Application Cache" being filled in. It will tell you if you have any bad links in your cache.manifest file, and you should fix those.

2. Once you're viewing your cacheable web page, click Refresh. This causes the browser to test that the manifest has good links, and again you should fix any problems you see.

3. Now that you've tested it in your desktop's browser, you can test on an ipad by pulling the link up in Safari and clicking the "Add to Home Screen" link. This will create an icon on your ipad home screen for your app. You can then click that icon to test that everything works. And then to be really sure it works, turn on Airplane mode in settings and test that you can still launch your app.

A note about testing in chrome: you cannot clear the application cache by running the standard "Clear Browsing Data". Instead, you should go to the magic url:  chrome://appcache-internals/ and click "Remove this AppCache" to clear it. On the ipad, if you go to Settings, Safari, and "Clear Cache" then it appears to clear out and invalidate your local application cache.

HTML5 Storage Wars - localStorage vs. IndexedDB vs. Web SQL

Currently, there are three competing approaches for saving serious amounts of data (i.e., persistently, and bigger than cookies) locally in your browser:

1. Web Storage
2. Indexed Database API
3. Web SQL Database

These names sure seem similar. But the implementations sure are different. Let's quickly summarize what they do, the PROs and CONs, and what I like best at the moment. Though I'm sure my opinions will age quickly as the technology matures.

All these technologies use the same-origin protection for data access (i.e., javascript can only access data from the url's domain that it was served from), which is fine and not a differentiator so I won't mention that below.

Web Storage

Web Storage, and specifically the localStorage part of it, is a really simple key/value persistence system.

PRO: Really simple API.

PRO: Already available in all major new browsers.

CON: No query language, schemas, really nothing you'd normally call a database. So it wouldn't scale well where you need to impose organization on a larger data set.

CON: They didn't put transactional safety into the standard. I don't think I can sleep at night with an app running that might have race conditions and then have the risk of corrupt data.

Indexed Database API

IndexedDB is basically a simple flat-file database with hierarchical key/value persistence and basic indexing.

PRO: If you're a NoSQL type of person, then this might fit the bill perfectly.

CON: Not yet available in most new browsers.

CON: If you wanted SQL, you're not getting it here. Though in the future, it might be a great building block for implementing a SQL engine for the browser.

Web SQL Database

Web Sql is basically sqlite embedded into the browser. I've used sqlite off-and-on for a few years. At first I was turned off by the name (excuse my superficiality) and by the architecture (just a flat file database). But after digging into it, I found that it's a rock solid platform, and great for production use, as long as you keep in mind its limitations. Its limitations aren't so much size (I think you can go at least 1 GB without a problem), but inherent in flat file databases (high levels of concurrency) and missing features (stored procs and other higher-end database features).

PRO: Fast and pretty feature-rich sql implementation (besides select/insert/update/delete, you can do joins, inner selects, etc).

CON: Available in Chrome and webkit-based browsers (Safari, etc.) but not Firefox or IE.

CON: The darn W3C Working Group has put a hold on the standard since they say they want at least two independent implementations of the standard, and there's only one so far, since everybody is using sqlite.

I wish the standards group would make a special case for sqlite and approve this standard -- it's in the public domain, so it's available to everybody in the world, no strings attached. Sqlite is a perfect fit for browsers - its limitations are not problems. Only one user is using a browser at a time, so no concurrency issues.  And no one wants high-end database features in a simple no-administration database. I think that's why both Android and iOS use sqlite for storage.

Bottom line

If you're only deploying on mobile platforms, then Web SQL is a no-brainer. Or if you're running on desktops and can require Chrome or Safari as your browser, then Web SQL is also for you. I wouldn't use the other two standards in any heavy-duty app at the moment.

Python on Android -- Easy as Pie

Here's what I did to get Python running on my HTC Incredible:

Step 1:  Install a barcode scanner, so that it's easier to install custom apps:  Install ZXing app

Step 2:  Install the SL4A application (Scripting Layer for Android) by going to the SL4A project home page and running the previously installed "Barcode Scanner" app (the title on the icon of the ZXing app) to scan the SL4A's barcode (or just click the link if you're reading this on your Android). This downloads a .apk file which you then run to install this app directly onto your phone, since it's not yet available in the Market. You might have to change your phone's settings to allow installing non-Market apps -- go to Settings / Applications and make sure "Unknown sources" is checked.

Step 3:  Install the Python interpreter:

1. Run app SL4A.
2. Press the menu button, then View, then Interpreters.  You should only have Shell at first.
3. Press the menu button, then Add, then select Python 2.6.2. After the .apk file for Python downloads, run it to install it.

Now you can edit and run Python scripts on your Android, and you have a lot of the Android functionality available. Here's my first test script:

import android

droid = android.Android()

droid.ttsSpeak('Hello World')

You can press Help to see the API that's available, it's a subset of the entire Android API.

Open Source Data Modelling - Power*Architect

There are a ton of half-finished or half-complete open source data modeling software projects out there. I've just been evaluating them for a client. At the moment, Power*Architect seems the best for my needs:

• open source
• multi-platform - Java in this case
• multi-database support - I need PostgreSQL and Oracle
• pdf export of ER diagram
• alive and not abandonware

And to top it off, it's actually easy to use! Take that, ERwin. To be fair, it doesn't have all the features of ERwin, but that's OK for me.

Announcing New Consulting Offerings

We've actually been doing consulting for a while, and now we're formalizing it into two product areas:

• PyGuys - Python and open source based solution consulting
• PyCOE - Python Center of Excellence, we help you set up a process or organization inside your company to encourage adoption of Python

Summary of Python Web Frameworks, Beginning of 2008

Let me summarize our recommendations for Python web frameworks:

• Pylons: for database-driven applications
• Plone (built on Zope3): for CMS (content management systems)
• Twisted: for multiple/custom network protocols or extremely high volume

Overall, the Python web community is seeing a lot of active development and evolution in solutions. Whereas the Ruby community has one go-to web framework (Ruby on Rails), the Python community has a plethora of choices. Luckily, a Python standard was created, PEP 333 aka WSGI (pronounced "whiskey" but stands for Web Server Gateway Interface), which provides a standard component architecture for Python web servers. All of the above frameworks support this standard now. This interface standard allows plug-and-play middleware.

We recommend Pylons for database-driven application development because internally it is built on WSGI. If TurboGears 2.0 development proceeds as designed and it is written on top of Pylons, then we will recommend it. Here are the middleware components that we recommend using with Pylons, and the features of each that we really love:

Database ORM: SQLAlchemy

• well designed: provides nice object interface to SQL tables, but also let's you get to underlying SQL if you want to
• really efficient: can load entire graph of objects in one SQL query - e.g., load item and line item details in one round-trip to the database

Templating: Mako

• fast: it's the fastest of the Python templating engines
• simple: you use standard Python in your templates, instead of custom tag libraries

Request Dispatching: Routes

• flexible: regular expressions with intelligent name binding to map tidy-looking urls to your internal object hierarchy

Form handling: ToscaWidgets

• no repetition: you model your form elements using Python objects, and use this model both for display and validation
• flexible: you can either use the default or specify your own template for html generation

Every category of middleware above has competition, but these are our current recommendations. Even though Pylons is a relatively new project, it is built on more mature components that are on at least their second major version.

Hey Mister, You’re Gonna Have to Rewrite that Web App

Good old blogger Joel Spolsky just wrote a great overview of the state of web applications. I'm inclined to agree with his prediction, that we're still in the midst of evolving the technology in web apps. On the desktop all apps have a consistent look and behavior but web apps have their own unique and different user interfaces. On the desktop you can copy and paste between apps, but not on the web except for a few words of text. I wonder how many years it will take for web apps to reach the level of design stability that desktop apps enjoy...

Someone Making Fun of the Consulting Sales Process

I couldn't say it better myself (from Services Safari weblog): My Thoughts on Improving Consulting Sales. I think I've seen all those things done during the sales process (but not by me or anyone under me!).

Housing Prices

Over the past few years, I've found that a suprising number of people are interested in investing in real estate. Everybody sees housing prices go up, up and up. But I just found a graph of housing prices in the US from 1890 to today, adjusted for inflation (hat tip to the Freakonomics Blog). In summary, housing prices have stayed mostly constant over the long term, once you adjust for inflation. I think people forget to adjust for inflation, and then they think that they made a lot of money by investing in real estate.

The most interesting thing to me is that over the last 10 years housing prices have doubled. That's no surprise. What is a surprise, however, is how many people think housing prices will stay at their current levels. I think this situation is similar to the Internet stock bubble. I remember just before the bubble burst, all sorts of people who knew nothing about the stock market were putting their savings in the stock market, because they didn't want to miss out on a sure thing. Well, that bubble burst, and I think this housing one will too, though at a slower pace since housing is less liquid than stocks.

Vernor Vinge's Technology Predictions in Rainbows End

Vernor Vinge is one of my favorite sci-fi authors. I just read his new book, Rainbows End, and I highly recommend it. It's set in the near term future (within 20 years), and it has a bunch of technological and social predictions I want to list here since I can't find them on the Internet (Vinge has a 50 min podcast available which discusses some of them). These aren't plot spoilers, but they are technology spoilers, so don't read this if you plan on reading the book:

• Affiliances - These are some kind of part-time work / large-scale special purpose corporations. I think the great idea here is that with the right software and network infrastructure, it should be possible to break down problems into arbitrarily small sub-problems which can be outsourced to other people anyplace in the world.
• Belief Circles - This is directed online communities on steroids. This is when online communities which have a common belief system (say fans of Harry Potter, Trekkies, etc.) spend so much time together online that they are able to accomplish things in the real world too.
• SHE (Secure Hardware Environment) - Following the trends of many modern governments to control encryption and other advanced or threatening technologies, the ultimate dream of government control freaks is that all computing hardware must be regulated by the government. And this government regulation requires that the lowest level computing infrastructure gives back door access to the government to everything that takes place in computers.
• Localizers - Imagine very cheap wireless sensing machines the size of a grain of sand (something like this but smaller). And if they're really cheap, and they're scattered in every square meter around the country, then you've got ubiquitous high-bandwidth networking to everybody everywhere. But bye-bye privacy.
• Contact lens overlays - The idea is that you can change what you see through your eyes with your contact lens (but in his podcast he says he might change it to retinal implants). Instead of painting your house, just have everyone in your family use the same color overlay!
• SM (Silent Messaging) - People can send instant messages to others without talking or typing -- I think the idea is that someone growing up using advanced computer interfaces would be able to interact with their computer interfaces without explicit or at least viewable muscle movements. Wouldn't this just be technological telepathy?
• Large Teams Acting as One Giant Brain - What if you had high bandwidth and productive communication between all of your team members, and the team members were experts in a large range of areas, wouldn't the whole team acting together seem like one god-like intelligence?
• Friends of Privacy - This is some group of unknown people on the Internet who publish subtle and not-so-subtle lies to make it difficult to trust the information you read about people online. In other words, anonymously-published large-scale disinformation.
• YGBM (You Gotta Believe Me) - This is technology that let's you manipulate people's minds through broadcast media. I think it's probably possible to control people, look at cult leaders for an example.
• Ubiquitous Support - Imagine a real-time Google Answers. Imagine if you're a programmer and you've just spent 2 days stuck on a problem, what if you could, immediately and in real-time, pay to get one hour of help from some expert programmer somewhere in the world?