Open source in the clouds and in the debates
We continue to see more evidence of the themes we discuss in our latest CAOS special report, Seeding the Clouds, which examines the open source software used in cloud computing, the vendors backing open source, the cloud providers using it and the impact on the industry.
First, as usual, we are seeing consistencies between our own research — which indicates open source is a huge part of today’s cloud computing offerings from major providers like Amazon, Google, Rackspace, Terremark and VMware — and that of code analysis and management vendor Black Duck. In its analysis of code that runs the cloud, Black Duck also found a preponderance of open source pieces, in many cases the same projects we profile in our report.
Indeed, open source software is an important part of the infrastructure, data and application layers of today’s cloud computing stacks with significant use of Linux, open source hypervisors KVM and Xen, open source data technologies such as MySQL, PostgreSQL, Hadoop, NoSQL and memcached and open source languages such as Java, PHP, Python and Ruby on Rails.
There will be plenty of users and customers content to use non-open source options that serve as the defacto standards, but we do see a move to higher-level, production and mission critical use, which represents continued commercial opportunity for open source and other vendors.
One of the more subtle effects of all this open source in the cloud, as covered in Seeding the Clouds, is the impact on discussions, debates and downright fights in the market. There is much scrutiny on claims of being open, technical aspects of open and what ‘open cloud’ means. A prime example is the Twisticuffs that have gone on between Simon Crosby of XenSource and Citrix, discussing OpenCloud and the response from Open Cloud Initiative co-founder Sam Johnston, who claims this is misuse of the open label.
We already saw open source playing a role in the discussions and debates about open clouds, open APIs and open data, and this latest confrontation is evidence that role continues to be significant. We still wonder though about the question of open enough as we contemplate openness in the clouds.
Submit your presentations for Collaborate 2011!
There are approximately 3 weeks left to submit your MySQL DBA-related presentations to Collaborate 2011, held in Orlando, FL April 10-14. Experience has shown that the best presentations are submitted well in advance of the deadline, so now is a great time to submit while you have plenty of time to create a good abstract.
Information and the link to submit abstracts are at http://collaborate11.ioug.org/. I posted a cheat sheet on how to fill out the Call for Presentations for MySQLers a while ago, and as always feel free to ask me any questions.
read more
Digg’s main competitor (Reddit) runs Cassandra but their VP of Engineering was fired for the decision to switch.
Apparently, Digg performed a big migration from MySQL to Cassandra and a big migration to their new Digg v4 architecture and now their VP of Engineering has been shown the door:
Ever since Digg launched its new site design, it’s been plagued with all kinds of trouble, not least of which is that it keeps going down. The problems with the new architecture are so bad that VP of Engineering John Quinn is now gone, we’ve confirmed with sources close to Digg.
In a Diggnation video today, CEO Kevin Rose explained some of the technical issues the site is dealing with and why it can’t simply roll back to the previous architecture. The new version of Digg, v4, is based on a distributed database called Cassandra, which replaced the MySQL database the site ran on before. Cassandra is very advanced—it is supposed to be faster and scale better—but perhaps it is still too experimental. Or maybe it’s just the way Digg implemented it (Twitter uses Cassandra, although not for its main data store, as does Facebook in places, but it obviously is not as battle-tested as it needs to be). Every engineer at Digg is currently just trying to keep the site up and running.
Some of this is political. Perhaps Mr. Quinn was excused for other reasons above and beyond this switch.
Perhaps he should have had buy in from other members of the team. Had Rose personally signed off on this migration it would have been tough to fire their VP of Engineering.
The technical aspects on this type of migration are VERY difficult. Not just because you’re moving from one DB to another but a lot of the polish, fit, and finish of your existing system tend to be taken for granted over time.
Newer databases don’t have this type of polish and you end up having to duplicate a lot of infrastructure that’s already present on the previous generation.
MySQL is definitely no panacea. You’re going to have pain either way. At least with some of the modern DBs you’re partially headed in the right direction.
One trend I’ve seen is for people to use the LAMP stack to serve websites but then to use Hadoop + Hive as part of their ETL setup so they can run reports and transform production data.
There is no solid bigtable implementation just yet. I wish there was but it doesn’t seem like we have one just yet.
Cassandra isn’t that bad of course. Reddit, Digg’s main competitor – is running Cassandra.
Seems like a strange thing to fire someone over. If you’re main competitor is running the same database the decision to switch certainly couldn’t have been too bad.
Zmanda @ Oracle OpenWorld 2010
If you are coming to this year’s Oracle OpenWorld 2010, please do visit us at Booth #3824.
We will have our backup solution experts at the show to discuss any of your database or infrastructure backup needs.
When it comes to backing up various products offered by Oracle, we have several solutions:
Amanda Enterprise: Backup solution for Oracle Enterprise Linux and Solaris based systems.
Zmanda Oracle Agent: Backs up live Oracle databases to Amanda backup server - which can store backup archives on disks, tapes, or storage clouds.
Zmanda Cloud Backup: Backs up live Oracle databases running on a Windows server directly to Amazon S3.
Zmanda Recovery Manager for MySQL: Backup solution for large scale MySQL environments.
We hope to see you at the show!
Flexviews 1.6.0-RC1 is released
Whats new in Flexviews 1.6.0RC1This is the first release candidate before the final release. If no major bugs are uncovered, then the next release will be the first GA release. Flexviews now has a test suite for all major features. The creation of these tests uncovered a number of issues which have been resolved in this release. All MySQL aggregate functions except GROUP_CONCAT are now supported. A special aggregate function called PERCENTILE is now also supported. The calculation uses a modified version of the GROUP_CONCAT based solution suggested by Roland Bouman for percentiles. This function should be considered experimental. Please report bugs if you find any.You can add indexes to enabled materialized views using SQL_API/add_exprAdding PRIMARY KEY indexes is no longer supported. All views get an auto_incrementing primary key. You can add additional UNIQUE indexes instead.There is an upgrade process from 1.5.3b (see UPGRADE and upgrade.sql)Significant bug fixesViews with aggregate functions but no GROUP BY columns now work properly for all supported aggregate function typesNULL values in GROUP BY columns are now properly supportedNULL values now work properly with distributive aggregate functionsThere is a wrapper script around run_consumer.php which can restart the consumer if it stops runningAs always, get it at:http://sourceforge.net/projects/Flexviews
News Feeds