Weekly Google Code Roundup: New Gears, GWT out of beta, and YouTube meets GData

By Dion Almaer, Google Developer Programs

You know the summer is ending when the kids are back at school. We had a raft of exciting announcements this week, starting with the web developer tools of Gears and GWT, and including the latest set of Google data APIs to join the family.

The Gears team announced a new developer release. The release you, the developers, to play with new APIs including some new Gears modules (HttpRequest and Timer), and the ability to support cross-origin work.

Google Web Toolkit 1.4 was released. This release is particularly important as the beta moniker is no more. This is a fantastic release but the team is continuing to make GWT better. At around the same time, theClassConnection went public, which shows you what someone who has never written a web application before can do with GWT.

Stephanie Liu of the Google data APIs team introduced us to the new YouTube GData APIs. Now you can search through YouTube's index and get detailed video, user, and playlist information in the form of GData feeds.

Featured Projects

The Google Zurich office has released an exciting new open source virtual server management tool called Ganeti. Ganeti is built on top of Xen and other open source software, and here at Google, we've used Ganeti in the internal corporate environment to facilitate cluster management of virtual servers in commodity hardware.

Gears In Motion is the latest database tool to sit on top of the Gears Database module. It allows you to visualize your local datastores in a new way.

Featured Media

Chris Prince of the Gears team took some time to discuss the new developer release.

Philippe Ombredanne of the Eclipse foundation came to talk to Leslie Hawthorn about the structure of the Eclipse Foundation, and how it participates in the Summer of Code program.

We take the keyboard for granted, but Jaewoo Ahn came to Google to talk about MobileQWERTY a simplified keyboard concept suited for the mobile form factor.

As always, check out the latest tech talks, and visit the Google Code YouTube channel.

Updates from the Latest Python Sprint

By Leslie Hawthorn, Open Source Team

Google was pleased to host last week's Python Sprint. From August 22-25th, over twenty developers in Mountain View and Chicago came together to improve next generation Python, also known as Python 3000 or Py3k. Many of the participants got their initial taste of Python internals at the sprint.

There was a flurry of activity at the sprint, and over 100 changes were committed - about five times the normal rate! The team got so much done that you can expect the first alpha release of Py3k in a few days.

If you'd like to participate in Python development or just learn more about the project, check out their general developer's mailing list or the Py3k development mailing list.



The Mountain View Python Sprint Team (starting with the back row, left to right): Larry Hastings, Tom Waite, Ero Carrera, Guido van Rossum, Collin Winter, Bill Janssen, Yuri Ginsburg, Thuon Chen, Christopher Burns, Keir Mierle, Neal Norwitz

(photo credit: Paul Dubois)

Ganeti: Open source virtual server management software released

By Roman Marxer, Ganeti team

Today we're happy to announce the first beta release of Ganeti, an open source virtual server management software built on top of Xen and other open source software.

Ganeti started as a small project in Google's Zurich office. We've been using it internally for a while, and now we're excited to share it more broadly under GPLv2.

Here at Google, we've used Ganeti in the internal corporate environment to facilitate cluster management of virtual servers in commodity hardware, increasing the efficiency of hardware usage and saving space, power and cooling. Ganeti also provides fast and simple recovery after physical failures.

Feel free to download it from http://code.google.com/p/ganeti and don't hesitate to give us feedback.

Cheers,

Ganeti Team

Google Web Toolkit out of beta as of 1.4 release

By Dan Peterson, Google Web Toolkit team

Removing the beta label from a product is a great milestone, and we're glad to report that the latest one to make that move is the Google Web Toolkit (GWT).

With the release of Google Web Toolkit (GWT) version 1.4, we'd like to give a shout out to the open-source GWT contributors that put in many hours of hard work to make GWT what it is today. We look forward to continuing to make GWT better.

If you are new to GWT check out the story and then read up on the specifics about GWT 1.4:

There's lots and lots of cool new stuff in GWT 1.4, so it's hard to know where to start. How about application performance?! This release includes several breakthroughs that make your compiled GWT code significantly smaller and faster. Many users are reporting that after a simple recompile with 1.4, their applications are up to 30% smaller and 20%-50% faster. And startup time in particular is now highly optimized thanks to a new bootstrapping technique and the availability of image bundles. To see the new hotness in action, try visiting the new-and-improved Mail sample a few times. It's darn fast the very first time you visit it, but subsequent visits are insanely fast. That's because, in addition to a fast initial startup, GWT code uses a clever caching technique to prevent applications from making unnecessary HTTP requests. As Joel Webber (Tech Lead of GWT Core Libraries) would say, "The fastest HTTP requests are those that do not, in fact, occur."

Hungry for more?

• Download GWT
• Details for upgrading to GWT 1.4
  
• Visit the GWT blog
• Join the discussions on the GWT Developer Forum

YouTube: Now with GData Goodness

By Stephanie Liu, Google data APIs Team

YouTube is the latest service to join the GData family. Now you can search through YouTube's index and get detailed video, user, and playlist information in the form of GData feeds. If you haven't built something with YouTube yet, now's a great time to get started! Here are some examples to give you ideas.

Search through the index for new 'puppies' videos (I subscribe to this query, doesn't everyone?):
http://gdata.youtube.com/feeds/videos?vq=puppies&orderby=updated

Search through lonelygirl15's videos for the season finale (maybe you missed it, or just want to relive the drama):
http://gdata.youtube.com/feeds/users/lonelygirl15/uploads?vq=season+finale

List all of NBC's playlists:
http://gdata.youtube.com/feeds/users/nbc/playlists

Ooh! They have a Heroes playlists (I love Hiro):
http://gdata.youtube.com/feeds/playlists/E0E4BC86E9D0ACB3

The full list of functionality can be found in the reference guide on our new home on code.google.com.

For the old-school YouTube developers, the migration guide has info on switching over, including the upgrade timeline (you'll have at least a year).
Head over to the YouTube API Announcement Blog for the rest of the details.

I'm looking forward to seeing new faces in our developer forum - feel free to stop by with questions or feedback.

Weekly Google Code Roundup: Reaching the Sky and Writing Offline

By Dion Almaer, Google Developer Programs

It has been a busy time recently. The Zoho team announced offline support for their Writer application this week, so we met at their offices and talked to them about their experience. This is our first video talk, but more are in the works, so head over to our new YouTube channel.

If you are a Mac developer you now have access to more of our APIs via the updated Google Data APIs Objective-C Client Library. You can now work with Google Code Search, Picasa Web Albums, and do more with Google Calendars.

Speaking of Google Calendar, we introduced Calendar Gadgets which allow you to add behaviour to your calendar via Gadgets. Some early examples include adding horoscopes, sudoku puzzles that get harder throughout the week, and the ability to keep up with the all important celebrity birthdays.

If you are new to Gadgets, Alan Williamson has written a nice introduction to creating a Gadget for the Google Desktop.

The maps world has been productive. The big news of the week is the ability to embed a Map in a YouTube like way. Now you don't need to code to be able to build a map, and place it anyway you wish.

This doesn't mean the API is slowing down. Richard Garland wrote about a new cluster zoom feature that ties DragZoom and Marker Manager.

Introducing Sky in Google Earth has gotten a lot of people excited. Looking down at the earth is great, but being able to sit on your back and look up at the stars is just what you want on a nice summer night. Now you can do just that.

Featured Projects

Who's Web maps out various Web 2.0 talent on a rich Maps API implementation.

Zoho Writer has gone offline... in a good way. Now you can keep some of your docs available for that plane trip. Read more.

Featured Media

I got into a nice conversation with fellow Googler, and EAI expert, Gregor Hohpe at MashupCamp. Listen to the conversation about enterprise Mashups and the Google Mashup Editor.

Salesforce developers came to our offices and gave an Overview and Q&A on AppExchange.

John Resig of Mozilla and jQuery gave a talk on Best Practices in Javascript Library Design based on his work on both the jQuery library, and the new FUEL library for building Firefox plugins.

Michael Still talked about Practical MythTV, which covered the powerful open source personal video recorder.

Leslie Hawthorn has made all of her Summer of Code podcasts available in ogg format!

As always, check out the latest tech talks.

Introducing Calendar Gadgets

By Michael Bolin, Google Calendar Team

You've seen them on iGoogle, in IBM WebSphere Portal, and now you can find them in your favorite online calendar -- that's right, now we're supporting Gadgets in Google Calendar! If you're a Scorpio like me, you can add this gadget that lets you check your horoscope from your calendar (don't worry, all the signs of the zodiac are supported). And if astrology isn't your thing, how about games? This Sudoku gadget was created by Alex Komoroske, one of our interns on the Docs & Spreadsheets team. If you add the gadget, you'll see that the daily puzzles get harder throughout the week.

Finally, if you like to keep up on celebrity birthdays, the birthday reminder will show you who you are idolizing that day:



But you don't have to be a Googler to create a Calendar Gadget -- we have put together new documentation to help you get your favorite Gadgets into Calendar. If you are itching to write some code, then you can use GData or ICAL to add some fancy Calendar Gadget events to your calendar. But if you're out of Red Bull and don't have the time for a late-night hacking session, then you may want to try our new "of-the-day" Calendar Gadget wizard, which makes it easy to take an existing Google Gadget and have it show up once a day, every day, just like the Horoscope and Sudoku Gadgets do.

Let us know how your experiences with Calendar Gadgets are going in the Google Calendar GData API developer forum.

Google Funds COLLADA Support for Mac and Linux

By Christian LaForte, Feeling Software

COLLADA, the standard 3D interchange format, makes it much easier to translate complex animated 3D models across Google Earth, Google Sketch-Up, Adobe Photoshop, Epic Games' Unreal Engine 3, Autodesk Maya, and many other applications. COLLADA integrates 3D geometry, textures, complex material, complex skeletal and facial animation, physical simulations, and many more aspects. With such power comes complexity: integrating COLLADA from scratch in your application is not for the faint-hearted. Luckily developers have created reusable libraries to simplify this process, including FCollada, which is open-source, well tested, can import all versions of COLLADA and has been integrated into many applications.

We've worked on development of FCollada over the past two years, thanks to support from Google's Open Source Progams Office. Until recently, the library was only offered for Windows; it is now available for Mac OS X and several Linux distributions.

Google's sponsorship also enabled us to distribute and support two additional tools for Linux and Mac OS X enthusiasts:

• ColladaMaya, a complete COLLADA translator designed for Autodesk Maya. It's now available for all three major platforms under the MIT license.


• Feeling Viewer, a reference viewer for COLLADA content, supports all standard COLLADA features. A stand-alone version of the viewer is available for free on Linux and Mac.

Google Developer Podcast Episode Seven: Mashups in the Enterprise

By Dion Almaer, Google Developer Programs

Using iTunes?

Gregor Hohpe is a Googler who has spent a lot of time in the enterprise. He authored the book on Enterprise Integration Patterns, and has been watching and helping the integration world.

We were both at MashupCamp and started to talk about how Mashups could be called EAI 2.0, and started to discuss the similarities and differences.

Gregor also wrote an article titled Google Mashup Editor and Yahoo! Pipes: Friend, not Foe, which does a great job of explaining the mashup landscape by example. He shows how to use the Google Mashup Editor, and how it can work with Yahoo! Pipes and other tools.

In this podcast you will hear about all of these topics in that special style that only a German can offer. There are some really fun analogies here. Listen out for the graffiti artists in the Mashup world.

Oh, and on behalf of two europeans, we apologize for saying "San Franciso 49ers" when we of course meant the Giants.

You can download the episode directly, or subscribe to the show (click here for iTunes one-click subscribe).

Dreamweaver Tools for Google

By Sean Owen, Google Mobile Search

We keep making more services available to mobile users; in addition to Google Mobile Search, you will find that Google News, Google Maps, and more have had a mobile presence for a while. So, we want to make it easy for developers to link to and use these services in mobile web sites. I'm therefore pleased to note that we've collaborated with WebAssist to add new mobile tools to their Dreamweaver Tools for Google plugin. The newly-released version 2.0 of this free tool includes, among other things, one-click access to embedding Google Maps, News, Mobile Search, and Send-to-phone in a page.

Plant a Seed, Watch It Grow: Improvements to GeoServer

By Chris Holmes, The Open Planning Project

Last year, Google's Open Source Programs Office funded the GeoServer Project to add support to output data to Google Earth. In the venture capital world, there is a notion of 'seed funding': putting capital into a new, usually risky, project to try out an idea and help it reach a state of sustainability. Google wanted to promote the idea of using 'Network Links' in KML to enable organizations to put large amounts of existing geospatial data onto Google Earth. They found fertile ground in the open source GeoServer Project, where the seed would not have to grow in isolation, but instead could flourish alongside other improving components.

GeoServer was started by a non-profit called The Open Planning Project (TOPP) in 2001, with the goal of making geospatial data more available through open standards - not just images but the actual data, the 'source code' of the map. This opens the information to enable analysis, modeling and user corrections. Today it is a vital open source project, with many outside contributors. The latest release (download) contains a number of new additions for Google Earth, which the community has helped shape and improve over the last year.

New features include the ability to easily customize placemark pop-ups from existing data, support for 'Super-Overlays', powerful time visualization, and automatic generation of legends. These are all available from a variety of data sources, including PostGIS, Oracle Spatial, DB2, ArcSDE, Shapefiles, GeoTiffs, and ArcGrid, with more being added every day by the community. There are also several related improvements, including the ability to overlay data dynamically on Google Maps, as well as GeoRSS and GeoJSON output.

Other Google Earth-related news: TOPP is participating in a testbed put on by the Open Geospatial Consortium (OGC) to help figure out what the next version of the KML specification may look like. The GeoServer team is very excited about KML becoming an OGC open standard, as GeoServer already implements the main OGC standards. For the testbed TOPP will build support for the new version of KML in to GeoServer and OpenLayers, an excellent AJAX mapping client. To follow and participate in the work being done on the next version of KML, subscribe and contribute to the ogckml page on del.icio.us.

Open Source Developers @ Google Speaker Series: Michael Still

By Leslie Hawthorn, Open Source Team

Tired of your current DVR? Prefer to DIY? If so, please join us for Michael Still's upcoming presentation on Practical MythTV . Michael, one of Google's Site Reliability Engineers and MythTV developer, will be joining us on Thursday, August 16th, to discuss this powerful, open source personal video recording software. Michael will also discuss some of the current challenges with obtaining television guide data in the United States and highlight some forthcoming features.

Like all sessions of the Open Source Developers @ Google Speaker Series, Michael's presentation will be open to the public. Doors open at 6:30 PM at our Mountain View campus; guests should plan to sign in at Building 43 reception upon arrival. Refreshments will be served and all are welcome and encouraged to attend. Michael's presentation will also be taped and published along with all of the public Google Tech Talks.

For those of you who were unable to attend our last session, you can watch the video of Raph Levien's recent presentation Lessons from Advogato.

Weekly Google Code Roundup for August 10th

By Dion Almaer, Google Developer Programs

In API and developer-product news...

ExtMapTypeControl 1.1 contains a smarter traffic button. In this week's release of API v2.86, there is a "changed" event for GTrafficOverlay that sends a boolean indicating whether traffic data is available in the current viewport. Using this event, Pamela modified the traffic button in ExtMapTypeControl to behave exactly as the maps.google.com button does. The example below lets you test out the new behavior - pan and zoom out to the other side of the world to see the button disappear.

The Google Mashup Editor has a couple of great new mashups including EarthQuake! which gives detailed information on earthquakes and plots their magnitude on a Google Map. If you are a Sudoku fan, Offline Sudoku will let you play away even when the internet connection isn't there for you, using Google Gears.

Speaking of Gears, Arjun Kalura has created a database sync module that shows the progress of a sync between a remote database, and a local one. The example also uses the worker pool to execute SQL in its own thread and hence keeping the user interface freed up to show the progress report on how the sync process is going.

Also, Uriel Katz announced a new release of GearsORM which adds basic introspection including a model that maps to sqlite_master table.

Around Google

We feel pretty generous with the amount of storage that we give out with our services such as Gmail and Picasa, but we realise that some users would like more so we now offer a way for you to buy more storage if you need it.

Privacy is important to us, and our users. In order to improve transparency we have launched our first experiment to explain basic privacy concepts via video on YouTube.

Joining the Open Invention Network: "You'll often hear members of our open source team say, "Every time you use Google, you're using Linux." It's absolutely true. Check a Google engineer's workstation, and you'll probably find it's running Linux. Do a search on Google.com, and a Linux server will return your results. Ever since Google got its start, Linux has given us the power and flexibility we need to serve millions of users around the world.

In turn, we feel a strong responsibility to the Linux community, and we're always looking for creative ways to put our resources in the hands of Linux developers. That's why today we became a licensee of the Open Invention Network (OIN), an innovative patent-sharing organization founded to create a legally protected environment for anyone who works with Linux."

You will notice that the search results now tell how when fresh content shows up via a small piece of metadata such as "13 hours ago". This is an instant indicator of freshness, which you may, or may not, have weight depending on what you are looking for.

Featured Projects

MapMSG lets you create an electronic note on a map. The note can be in the form of smoke, crop circles, and more.

Walk Score is a mashup that uses the Maps and AJAX Search APIs to show you how friendly your location is for walking.

Google Tech Talks

JSR-305: Java annotations For Defects: This talk describes the current status of JSR-305, Java annotations for software defect detection. This JSR will define several standard Java annotations for properties such as @Nonnegative and @Nonnull that can be used to document your design intentions in a way that be interpreted by multiple software tools.

Open-source-based high-resolution cameras: Andrey Filippov explains the designs and applications of Elphel, Inc. intelligent, network-enabled cameras based on open source hardware and software. Google currently uses Elphel cameras for book scanning and for capturing street imagery in Google Maps.

View more tech talks.

Optimisation data for HTML5 parser implementors

By Ian Hickson, editor of the HTML5 specification

Last month, just before I left on vacation, I posted three sets of data to help implementors of the HTML5 parser specification optimise their code. There are several implementations coming along, for example those that are part of the html5lib project and the one behind Validator.nu.

The three sets of data that I posted are all derived from parsing several billion documents from Google's Web search index using a parser I wrote in Sawzall.

The first set of data gives the relative aggregate distribution of invocations of the "in head", "in body", and "in table" insertion modes, for each of the insertion modes. This allows implementors to determine, for instance, that invoking the "in body" code while in a cell must be very efficient, while invoking the "in body" code from the "after frameset" code need not be as efficient, in case the implementor has a strategy that optimises one at the cost of another. See: documentation, data.

The second set of data gives the relative aggregate distribution of tokens for each phase/insertion mode pair. This can help implementors that are using a cascade of if statements decide on the right order for their statements. For instance, the most common token type seen in the "in body" insertion mode is character data, and the second most token is the start tag token for an a element, but the isindex start tag was almost never seen. This tells implementors that they should check for characters and a start tags long before checking for isindex tags. See: documentation, data.

The last set of data examines the number of attributes per element. It allows implementors to decide on the optimum memory allocation strategy for attributes. For example, since most elements have 9 or fewer attributes, the data structure that stores attributes can be optimised for simply having 9 attributes, using little memory, and if an element has more than this number of attributes, the implementation can switch to a separate implementation that is more memory-heaving but is optimised for large numbers of attributes. See: data.

I hope this data is useful!

Weekly Google Code Roundup for July 30th to August 3nd

By Dion Almaer, Google Developer Programs

It feels like summer started a blink of an eye ago but we have reached August already. The summer months tend to be slower, but it doesn't seem to be the case so far at Google.

In API and developer-product news...

Create calendars with the Google Calendar Data API.

The Google Calendar data API now supports two new read/write feeds that allow you to manage a user's list of calendars. One feed lets you create and delete calendars, while the second feed can be used to add and remove subscriptions to existing calendars.

New API: It Slices! It Dices! It Uploads Your Docs!

Not only can you create new calendars, but you can create, import, and manage your Google Docs and Spreadsheets.

Google Mashup Editor, built with GWT

The Google Mashup Editor is a high profile, complex application that was written with GWT. Rich Burdon of the GME team discusses the rationale behind the GWT tool choice.

Looking for somewhere in India, Hong Kong, Taiwan, Singapore, or Ireland? Just geocode it!

Over the past few weeks, we enabled geocoding in the API for India, Hong Kong, Taiwan, Singapore, and Ireland. That means there are potentially 1.1 billion more users that might now be able to locate themselves on your map - that's about 1/5 of the world's population. (Pamela utilized GoogleLookup functions in a google spreadsheet to do the calculations.)

Chris Schalk has a detailed article introducing the Google AJAX APIs. The articles does in depth on all things Ajax.

Around Google

Microformats in Google Maps

We're happy to announce that we are adding support for the hCard microformat to Google Maps results.

How long will it take at rush hour?

The Maps team has added estimated timing on directions depending on rush hour.

Computer science resources for academics

Because we know that between teaching, doing research and advising students, computer science educators are quite strapped for time, we've recently launched a site called Google Code for Educators. While you may have previously heard about our offerings for K-12 teachers, this new program is focused on CS topics at the university level, and lets us share the knowledge we've built up around things like distributed systems and AJAX programming. It's designed for university faculty to learn about new computer science topics and include them in their courses, as well as to help curious students learn on their own.

Featured Projects

The Google Singleton Detector, or GSD, is a tool which analyzes Java bytecode and detects our different types of global state, including singletons, hingletons, mingletons and fingletons.

GWT Ext allows you to Ext components from within your GWT applications.

Google Tech Talks

Open Source Speaker Series: SilverStripe CMS

Advanced Topics in Programming Languages: Java Puzzlers, Episode VI

Hardware/Software Hacking: Joining the Real and the Virtual

LLVM 2.0

View more tech talks.

Podcasts

Google Developer Podcast Episode Six: The Hibernate Shards Open Source Project

We got to discuss the newly open sourced Hibernate Shards project with two of the core team that worked on it.

The Toronto Connection

For our latest podcast, we spoke with four students who are working together on their Summer of Code projects. Three of the students attend the University of Toronto, but Jeff Balogh visited the university at the start of the program to do some project planning with David Cooper, so he's an honorary UT alumni; both David and Jeff are working with mentors from the Python Software Foundation.

Google Developer Podcast Episode Six: The Hibernate Shards Open Source Project

By Dion Almaer, Google Developer Programs

Using iTunes?

Max Ross and Maulik Shah were part of a core group that worked on the recently open sourced Hibernate Shards project.

In the podcast you will learn:

• What sharding is and what it means in the world of Hibernate
• How the word "shards" is common at Google (the equivalent of "smurf" in The Smurfs)
• Why you would want to shard your data to give you increased scalable performance
• How the Hibernate Shards project doesn't mess with the core APIs, allowing you to add sharding unobtrusively
  
• What you need to think about if you want to shard your data, and how you can design a schema that has a dimension that is easily sharded. This includes designing without complex relationships.
• How you could create a crazy project that shards data across multiple databases (as in, one mysql, one Oracle), but that would be crazy
• The various strategies to define how you retrieve your objects across the distributed data store
• How this compares with horizontal partitioning at the database level itself (e.g. new features in MySQL, PostgreSQL, and others)
• And much, much more.

For more information check out the Hibernate Shards homepage and the Google Group for discussion.

The new release that was mentioned in the podcast just went live. Congratulations to the team.

Start listening now

You can download the episode directly, or subscribe to the show (click here for iTunes one-click subscribe).