The Everyday Scalability Blog

It’s a species that likes to talk more than it likes doing things. He’s always afraid of not getting cheated by people. The reason is that he is probably cheated all the time. And the reason for being cheated all the time is probably because he makes all the wrong choices about people.
The negotiations become excessive either before the project begins or after it’s completed – or should we say that the original requirements have been completed. The latter is a real killer because you have already put so much effort and/or time and/or money into it and you see that it’s slipping through your fingers. The first is also annoying but you still have a choice of getting out before it’s too late.
These people are the reason for which I think at least twice before embarking on some projects. These people like getting everything for very little. They don’t care how much you work, they don’t care what you do; they just know that you should do it all for them regardless of what the project was about and when you do it all for them they’re still concerned that they might have been cheated – it just might have been too easy for you.
So the bottom line is watch out for over-negotiators. Get out of it before it’s too late. Otherwise you will just loose time and money.

That was the question. At least for one project I have worked on. Contrary to Hamlet I have found an answer to this question but I have also learned that finding the answer does not (always) help. For those of you who just want to know who the killer is the answer was yes – but your mileage may vary.

For those of you who understand that the journey is more important than the prize you must know that the journey was pretty difficult. The main reason for that was the one thing I don’t like in connection with technology or science: fashion. Unfortunately fashion is a key factor in science and technology and if you don’t agree just think antibiotics: when they were discovered it was a truly remarkable achievement and they started being used with great joy; if you were ill, hungry, thirsty or just feeling bored you could get some antibiotics to cheer you up. Nowadays it’s sometimes more difficult to get some antibiotics than it is to get cocaine. How did that happen? It was just another fashion that came and went.

The best part about fashion in science and technology is that it’s always sustained by a rather complicated, logical and witty technical argumentation. That’s what happened on that particular project I have worked on: someone had the idea that relational databases are obsolete. They even talked to some guys that were specialized in databases, scalability or whatever that was; anyway they were obviously very smart people unlike the database supporters. I unhappily and unwillingly was of course a database supporter on that project.

When you try to support a point of view like this you just take the weak points of the adversary and maximize them and get all your weak points and minimize them. Then you add some catch phrases preferably with famous examples such as “Twitter doesn’t scale”. It was a mere reminder for all those people still trying to recover relational database supremacy that some people much bigger than you were brought down by this unworthy database machinery. And sustaining something fashionable you will only find cheerful people to support you just because it sounds good.

But what do you replace the relational database with? Well… let’s not hurry things and jump important steps. After all it’s the journey that matters the most. First you should try to search on Google for “scalability” or “scalable” or whatever comes into your mind. You will surely find lots of projects that are designed to be scalable or “with scalability in mind”. Then you make a list of all the names of those project and can use them in your next arguments. They might not be “exactly” what you want but they will surely make you look good because this shows that you are a true specialist: you know everything about this field.

The result for that project was waste of time and money and poor performance. In the end they reluctantly agreed that I would somehow shard the database. I did it in a hurry just so we would have something that would work. We now had a “hybrid” solution that “just worked” but which would hardly scale and which had pretty bad performance. Nevertheless the middle management reported good results not because of my “temporary” solution but in spite of it. The wasted time was regarded as “research and development” time with an emphasis on “research”. If there was ever any wasted time or money at all they were surely not responsible for it.

If you understood that database sharding is a must for your project then you should probably read it all again. I’m not an advocate of sharding or of relational databases or of any particular technology at all. I’m an advocate of honesty, good design and architecture right from the start and of the right technology at the right place in the right amount. That’s what I always try to do on the projects I work on. Unfortunately this is not always understood or appreciated – the “fashion people” are just more appealing and who’s more appealing has more chance to be considered to be right by the people that lack technical expertise. I lost time, money and credibility by not compromising and by not becoming one of the “fashion people” but I just hope that I won’t change in these aspects.

I have found out just recently about ma.gnolia. I expected to find some social bookmarking site but instead I was presented with a mysterious announcement and a couple of links to some blogs that would explain something about how the entire site quickly disappeared at some point in time. I was curious to find out more about it and chose the video blog link. A number of things caught my attention:

• Bad publicity is almost always better than good publicity. I found out about this project when there was nothing left of it but it’s true that this is a coincidence. On the other hand I would have left the site pretty quickly if I hadn’t been made curious by the misfortune of those poor people. So the bad publicity made me interested in the project and I’m most probably not alone in this situation. Otherwise the tabloids wouldn’t be so popular and we would all read science magazines only. This thing made me sad because I have always tried building positive things that somehow struggle to make it to the top; I could have added an extra-negative thing to them so I could have made things work out better for me but my sense of morality has always stopped me.
• Good project ideas and good technical expertise almost never see eye to eye until it’s too late. In other words most of the entrepreneurs don’t have the technical knowledge but even worse they usually fail in finding somebody with the required technical knowledge while staying withing the budget limits. This is sad because it’s a loose-loose situation: the projects loose expertise and have a good chance of failing or loosing money and the people with the expertise (such as me 🙂 ) loose because they have a hard time finding projects that need their skills.
• Perception and image became even more important than the product itself nowadays. Larry talks about how people thought that the project was bigger than it actually was and this probably led the project to a bigger success. Unfortunately the opposite is usually true for other projects: they loose because they cannot project an image that would allow them to get that bigger success. That’s why the advertising and PR industry is thriving; that’s why there’s so much money involved in this; that’s why so many of the internet services come for free – because when we buy a pen or anything else only a fraction of the cost is the pen itself and the rest is for the image we want about the pen which then pays for our free internet search engine, our free email service and so on. It is sad because we then complain about the quality of the pen and we shouldn’t – because we preferred the image instead of the true quality. Otherwise why wouldn’t you pay for your email services or your internet searches?
• As a result the projects such as ma.gnolia often have a hard time becoming profitable even if they are successful. It is sad and dangerous that so many internet projects must rely on advertising as their sole income.
• Why on earth would you ever use a Mac as a dedicated web server or database server or any other dedicated server? I couldn’t think of anything that would justify the extra-cost compared with a simple linux-based x86 server. This kind of sums up what I’ve written above: the lack of good technical advise or the image that becomes more important than the product itself. Apple products rely very much on brand and image. They are usually quality products but they are pretty expensive. The extra-cost compared to some no name products that provide similar quality goes on the Apple image: advertising, PR etc. Apple does not have users – Apple has fans.

All of the above can be applied to other projects as well. They are common snapshots of the internet start-ups of today. I hope that more people will acknowledge such things and this way we’ll make the internet a better place.

… can’t live with them but you surely can’t live without them. They are the driving motor of the internet economy (or one might argue that they drive the entire economy). They also spark the innovation: if you take the most famous websites that come first in your mind you will see that most of them have once been humble startups. And not only have they started very low but they also rose very fast.

But things aren’t so nice when you step into the startup world. You soon discover that almost every person dreams a startup dream and most of them even try to pursue it. Those people often have little knowledge about the technology needed and/or the business they’re trying to get into. All of them start with an idea. The problem with the idea is that most of the time it’s either (technologically, legally, morally, financially etc.) impossible to implement or if it’s an “original” idea there are probaly hundreds of people that already had that idea and tried to implement it but you haven’t heard of them yet.

I have actually seen projects that were looking for people to implement either a better Google or a new operating system that would run programs from all other operating systems but without having the other operating systems problems. Not bad ideas per se but the real idea is the implementation plan which is usually non-existent . These people even had a decent budget behind them and maybe that’s a even bigger problem: the fact that these crazy ideas actually get financed sometimes to the detriment of some good ideas.

It’s all about times and fashion. If it’s a good time for investments a lot of projects get financed even if most of them are bound to fail. If there’s a recession many projects die because of lack of finance even if they are viable. The finance people have even less knowledge or interest in technology than the entrepreneurs; they are usually more interested in markets, trends, hypes, coolness – the bottom line is they don’t care what it is as long as it sounds cool because they think that everybody (including your client) thinks the same way. The same “fashion” thing applies to project ideas too: a few years ago everybody wanted some kind of Youtube clone. Nowadays everybody wanta some kind of local YellowPages with Google Maps mashups.

The third thing after financing is getting the work done. As we have already seen the entrepreneur doesn’t know how to do it so they have to hire somebody. And because of the tight budget they usually preffer the cheap option: either project bidding websites, outsourcing companies or offshoring their business themselves. The result is usually some guy(s) in some very remote part of the world that do the job for very little money. Sometimes the entrepreneurs are lucky and get a very well done job for very little money but sometimes they’re not. It’s really a lottery but it also depends on how you buy your ticket.

Usually offshoring the development yourself provides much better results but it’s more expensive – so it’s not for everyone. Outsourcing companies sometimes provide good quality but it really depends on the company and the complexity of the project: if you want something which is technologically advanced they have a good chance of failing because they don’t have the necessary expertise. Getting to individuals or project bidding sites usually works only for fairly small tasks – it’s really a circus out there. Unfortunately many times there’s no corelation between the quality of the work and the cost in any of these methods; that’s the risk you have to take when you use outsourcing or offshoring. The trouble is that many people don’t know the magnitude of this risk while some others overestimate or missinterpret the risk – either way they loose.

… have I decided to start this blog? Simply because I have been passionate about computers for the last 20 years or so, I have written programs in over a dozen programming languages, I have worked extensively with several OSes but most importantly because during the past few years I have worked on scalability issues (yes – on “the cloud”) on several projects and I have been astonished by so many things I encountered during my work. There are many technological resources out there on the net about scalability, cloud computing and all the other stuff that is so popular nowadays. There are also many blogs, news, papers and columns about the business: the money, management, marketing, targeting and so on and so forth. However I have not found much on the net about the every-day life of the guys involved. And as in any bussiness (probably) that’s where the surprises (good or bad) begin.

I thought of sharing these “surprises” with everyone else hoping that they will be useful to other people: those who want to build a project and need the mighty computing power or those who want to make a living of providing the computing power to bussinesses. I scaled myself from a humble programmer to a scalability / cloud-computing expert or consultant. This provided me with a deep understanding of how things work or how they should be done but it has also led me to political issues. Who wins or looses from this you will see in my future posts.