The Secret Source of Google's Power

The first time i saw this, i just read the first paragraph and assumed it was going to be an anti-google rant and switched off. Now that everyone is pointing to it, i'm persuaded that this is one of the most important discussions on designing scalable web stuff. And Akamai shares the secret sauce. A discussion on GMail design. PS: PHP5 RC2 is out Discuss (Join / Login first) Edit   permalink: #

Discuss (11 responses) (Join / Login first) Edit   permalink: #  

dot Net and all that

A few months ago, we bought a copy of Visual Studio.NET 2002. Its been lying around the office (shh, don't tell my colleagues otherwise they'd brain me for wasting money), and i have only found the time to do some serious testing these past few weeks. This is not a review, but my impressions of Visual Studio.NET from a PHP coder's perspective. I also have prior experience with M'soft developer tools, having used Visual C++ since 1994, and VB and VFP since 1997. I still code in all the above languages. The first time I tried the Visual Studio.NET IDE, I found it a bit distracting as there are so many context sensitive help and information panes to sort out. There is a Server Explorer, Solutions Explorer, Properties Viewer, Several Help Viewers, the graphical form editor, the source code editor, dozens of toolbars, a scripting engine - whew! A bit like the confusion you feel the first time you try out Dreamweaver MX, only much worse. After more testing, I realized that the main problem is that the help and coding windows share the same screen space (unlike previous version of Visual Studio, where Help only ran as a separate app), so there just isn't enough space. Showing both help and code editing features in the same screen is bad user interface design. Fortunately this can be fixed by opening multiple copies of VS.NET, or hiding all help screens in VS.NET, and only use the external help application. JScript I was hoping to find good support for JScript.NET in Visual Studio, as this is what i use in my ASP programming. Sad to say, Microsoft has decided to only support VB, C# and C++ in the IDE (and Java in VS.NET 2003). You can still code in JScript, but you have to use the command-line tools yourself. I suspect this decision was motivated by business considerations, as from a technical viewpoint JScript has many features that make it more flexible and powerful than C++ or C#. Test Project Then I had to decide on a test project. In our current work with PHP for intranets, we find that a web browser is not rich enough a client platform. So we are building Windows applications that use XML-RPC and IE Web Browser control embedded in the app. This combine the flexibility and ubiquity of the web with the rich UI and ActiveX components of Windows. Creating such an application in C# seemed like a good R&D project. Visual Studio invisibly generates a lot of code for you as you manipulate the form graphically. When you add an ActiveX control to a form, the appropriate COM to .NET bridging code is added for you automatically. Very nice. But you still have to understand what Visual Studio is doing in order to customize the code. C# However my first attempt with C# was a failure because C# does not support default function parameters. You have to learn what every parameter of the IE WebBrowser control does to use it; so we couldn't simply adapt our existing VB/FoxPro code to C#. This was when i realized that though C# might be a nice language, VB.NET seemed to be the more practical choice. VB.NET In a few minutes the VB app with IE embedded was working. I easily created buttons on the form that allow you to navigate to specific pages, and we were able to retrieve and parse the HTML received (we embed control information from the web server in invisible HTML comments). I have found previous versions of VB had very poor standard function library. No regular expressions, no explode, few ways to manipulate dates unless you convert them to strings, no md5 support and the like. The new .NET Framework provides a huge class library with all these functions. This is an enormous improvement. ADO.NET The next step was to try out database connectivity. Now i am aware that my first attempts which i will describe below might not be the best way of coding, but I am following the sample code autogenerated by Visual Studio. In Visual Studio, you can browse an ODBC data source and drag and drop a table into a form. This will generate two objects: SQLConnection and SQLDataAdapter. The SQLConnection is the same concept as a PHP connection resource. The SQLDataAdapter is the object that holds the (compiled) SQL statement. You call the SQLDataAdapter() to generate have a recordses in ADO.NET. The recordset can be stored in an XML structure called DataSet . Then you can create a datagrid and bind the data to the grid by setting the grid's datasource property. The little gotcha's for a newbie to create a datagrid in a form are that although Visual Studio generates all the objects, operations such as running the SQL, the insert and update logic still have to be manually coded. For example, you have to manually type the following code yourself to execute the SQL embedded inside the SQLDataAdapter: SqlDataAdapter1.Fill(Me.DataSet11.TableName) As the author of a PHP forms engine, I find all this strange, because our phpLens engine will automatically perform the SQL query, insert and update the data grid for you provided you specify the primary key. This also compares unfavorably with Dreamweaver, which allows you to do all this without any coding. Alternatively, if you don't require data in XML, but prefer something similar to the way PHP does things, you can use SQLCommand/DataReader objects. sqlCmd = New SQLCommand('select col1,col2 from table', connection); dataReader = sqlCmd.ExecuteReader(); while (dataReader.Read()) Console.WriteLine("\t{0}\t{1}", myReader.GetString(0), myReader.GetString(1)); There are nice improvements in ADO.NET if you are using DataAdapters/DataSets (better disconnected recordsets, XML support, multiple table support), but they appear to be largely ignored by mainstream ASP.NET developers. IBuySpy is a website created by Microsoft to showcase ASP.NET best practices. The IBuySpy white paper says: One of the most common data access related questions that comes up concerns the difference between the DataReader and the DataSet. "When should I use a DataReader and when should I use the DataSet?" IBuySpy uses the DataReader almost exclusively. This is because, as in most Web applications, the majority of the data in the IBuySpy store is read only and we were interested in very fast access to the data. This is the most ideal solution for the majority of scenarios, except for cases where you need functionality beyond a forward-only reader. ASP.NET Events ASP.NET also has an interesting event driven architecture where you can define various event handlers which are triggered on the server-side, called postback events. You can click on nearly any HTML element in your web browser, and if configured to send postback events, it will trigger a POST to the server. This is certainly useful in an intranet environment and is obviously similar to the Windows event model, but care needs to be taken on the Internet when latency is an issue. Report Card A The .NET Framework provides a nice library - a big improvement over the limited libraries of previous programming languages and the low-level nature of the Windows API. A VB.NET. Massive improvement in power, not a boy-toy any longer. Though I prefer is C-style language conventions, I have to admit that VB.NET is really the best language in Visual Studio.NET. B Visual Studio.NET IDE -- ok, but very confusing compared to the previous version of Visual Studio due to the numerous new options. Definitely design by committee. B ADO.NET -- on first looks, there are nice improvements, which later turn out to be rarely used. C Visual C# -- the language is overhyped, not as powerful as VB.NET nor JScript. No Eval(), no default parameters, unable to switch to 4GL mode (automated type conversion), another clone of Java. C# is no more than a clever tactical move by Microsoft to encourage Java and C++ programmers to move to .NET. F JScript.NET -- a really good language with much more functionality than C# being killed by Microsoft because they won the browser wars, and Javascript originated from their enemies. General Assessment In comparison with PHP, I would say that .NET is more flexible (code in several programming languages, unified UI for the web and windows forms, better component model, will run on Linux if mono is allowed to prosper). However I don't find the web forms that easy to use. Though it generates some of the source code automatically, you have to write some of the UI code and all the business logic yourself. PhpLens has a state engine that is able to do this automatically. With no coding, all the logic and UI for search forms, browse screens, edit and new record forms with validation and must fill checking is done for you. Also ASP.NET's functionality is more for programmers developing intranets than internet web sites, as it's form oriented approach is a turn-off for web designers. The result of this assessment came as a surprise to me. When I first started my evaluation, I thought that C# would be a Java killer - instead it just turned out to be another Java clone with all its limitations, and C# is not even as powerful as VB.NET! I had a pre-conception that the .NET designers had learnt from Java/PHP/Cold Fusion, so they would have made many improvements in web development. I nearly believed Sterling Hughes when he said that he was depressed about .NET's superiority. It is true that ASP.NET is superior if you are converting a Windows program written in VB to a Web-based one. But when you stop reading the long long feature list and the "gee-whiz it's wonderful" articles and actually try out Visual Studio.NET, you still get the feeling that M'soft wants the web to go away, and with the Visual Studio tools are trying to make the web look as much like Windows as possible. I think the reason that many of us have found the web attractive is precisely because it isn't like Windows. When I want to code Windows, .NET is cool. But when I want to do web stuff, .NET is still about looking like Windows and graphical controls embedded in forms, stuff that just gets in the way when you're trying to code a web page. In preparing this article, I reviewed what Joel Spolsky had to say about .NET. It shows his human side. In 2000, he said "Microsoft's latest announcement, called Microsoft .NET ... proves that something has gone very, very wrong in Redmond." But in 2002 he changes his mind, ".NET appears so far to be one of the most brilliant and productive development environments ever created." I think the reality lies between these two passionate opinions. The default PHP install does not come bundled with an extensive set of form processing tools. However there exist good 3rd party tools for creating forms with PHP. These include commercial tools such as Dreamweaver and my company's product: phpLens, and open source code frameworks such as LogiCreate and EZPublish. eWeek reviews .NET after 3 years.

Efficiently Updating Web Sites on Clusters

At the time, our web site ran on a 2.2.x Linux distribution on a dual Dell 2450. Its performance was rock solid, but we were uneasy relying on a single machine for our entire business. We determined that the answer was to replace the single server with a cluster. For less than the cost of the Dell, we built a cluster of four 1U one-processor machines. The cluster's performance was excellent. By stripping down Apache, we could support at least 400 downloads over HTTP and still have a responsive site. This left one problem: we needed to be able to update the site often without affecting the performance. -- M. David Minnigerode Discuss (Join / Login first) Edit   permalink: #

The Eight Fallacies of Distributed Computing

Essentially everyone, when they first build a distributed application, makes the following eight assumptions. All prove to be false in the long run and all cause big trouble and painful learning experiences. 1. The network is reliable 2. Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn't change 6. There is one administrator 7. Transport cost is zero 8. The network is homogeneous -- Peter Deutsch Discuss (Join / Login first) Edit   permalink: #

Interviews With Martin Fowler

In this six-part interview, which is being published in weekly installments, Fowler gives his views on many topics, including refactoring, design, testing, and extreme programming. In Part I, Fowler makes the business case for refactoring and testing, and describes the interplay between refactoring, design, and reliability. In Part II, Fowler discusses design principles of avoiding duplication, separating presentation and domain logic, being explicit, and describes how refactoring depends on code ownership. In Part III, Fowler differentiates between planned and evolutionary design, suggests that focusing on superficial problems can lead to the discovery of substantial problems, and claims that doing a good job won't slow you down. In Part IV, Fowler discusses design decay, flexibility and reusability versus complexity, four criteria for a simple system, and interface design. In Part V, Fowler describes the unhurried quality of test-first design, defines monological thinking, and distinguishes between unit and functional testing. In this final installment, Fowler describes how to balance maintainability and efficiency and create tunable software, and discusses the role of patterns and the Agile Software Manifesto. Discuss (Join / Login first) Edit   permalink: #

Creating Reliable Software (pdf)

This is a presentation detailing how to go about software walkthroughs, code inspections, correctness proofs. Here's the google html version. What's interesting to me (you have to go to the last page) is that using mathematics, Peter Naur proved that a specific algorithm was correct. Unfortunately the specifications were flawed so the algorithm would never have worked, so Peter Naur actually proved that garbage in, garbage out. Once again, this shows that code reliability issues are so closely interwined with people issues that you cannot talk about reliable software without talking about reliable developers. Discuss (Join / Login first) Edit   permalink: #

Developing Reliable Software with Scripting Languages

In this interview of William Guttman, a professor of economics and technology at Carnegie Mellon University who is director of The Sustainable Computing Consortium, I was struck by this statement: Q: Are certain types of products or software vendors worse than others in terms of their record on quality? Guttman: I have a colleague who says bugs are agnostic. No matter the type of application it is, you can count on finding 30 bugs per 1,000 lines of code on average. I then found Prof. Thomas Huckle, saying much the same thing: INTEL: no more than 80-90 Bugs in Pentium. Standard Software: 25 bugs per 1000 lines of program. Good Software: 2 errors per 1000 lines. Space Shuttle Software: < 1 errors per 10000 lines. Example Handy (Cellular Phone): 200 000 lines of program: up to 600 errors. Windows-95: 10 Mill. lines: up to 200 000 errors. This seems to be a pretty good justification for using scripting languages. Of course, we still need to plan and be aware of the software engineering issues in developing reliable software. Here are a few things that occured to me. I'm sure you can think of more: 1. Scripting makes it too easy to slap up a piece of code quickly. I would suggest you slap your design first. Use a design methodology. The use of popular techniques such as OOP (or structured programming) provides a more disciplined approach to developing software. Informal evidence suggests that using object-oriented programming forces the developer to be more careful in the design of the software. Scripting languages that provide good OOP facilities should be encouraged. 2. Scripting languages have more options than bisexuals because they have polymorphic types (I have never liked the term weak-typing because it suggests something inferior. I prefer polymorphic typing -- it has the cachet of a poly-syllable adjective :-). One criticism of this approach is that very little parameter checking is done by the compiler. To me this is a benign vice, but I agree that with large teams where well-defined interfaces are important that this is an issue. I have three suggestions here: (a) Use classes to store important variables, because then you can use set and get methods for type-checking, and pass instances of these classes around instead of variables. To promote heterosexuality i have this example: function Reproduction($father,$mother) {   $sperm = $father->GetSperm();   $egg = $mother->GetEgg();   return "$sperm meets $egg"; } In the above example, we have type checking of $father and $mother (at run-time), because $father has GetSperm() and $mother must have GetEgg(). (b) Use intelligent IDE's that display the function prototypes as you type (what is the best one for PHP - does anyone know?) (c) Programming by contract. Techniques such as this provide additional assurance about the quality of the code (note that programming by contract is as applicable to C or Java or Fortran as scripting languages). See also what Bertrand Meyer has to say. 3. Scripting languages need fewer raging egos and more team players. What i mean by this is that scripting languages need to balance the needs of hackers who want a quick fix with software managers who need good support for team development, in particular offering effective ways (methodologies?) for separating code and content, data and presentation, and neat software engineering concepts such as data hiding and namespace support. 4. (added 22 Dec 2002) If you don't work up a sweat you're not exercising your code. Scripting allows you finish things faster but reliable code doesn't come for free. You have to work for it. A Ruby program might take 3 days while a Java or Pascal program could take 6 days. It is possible that the Ruby script is less reliable because less thought was given to it. But how about a Ruby script that was written in 3 days, and then another 3 days were spent in integration and requirements testing? The Java or Pascal program would have only had some basic unit testing after 6 days. Of course we don't want you to exercise the code till you have a heart-attack; there are diminishing returns to testing, so you have to decide when to stop and not go overboard. In conclusion, as Tom DeMarco pointed out in his book PeopleWare, it is not technology that prevents software success, but people. So higher level tools that aid people in their work are to be encouraged. Looking down at scripting languages because it lacks this or that is short sighted because it is the now and the future of computing. References: An empirical comparison of C, C++, Java, Perl, Python, Rexx, and TCL. Also has a short followup to this article. Scripting: Higher Level Programming for the 21st Century by John K. Ousterhout. Discussion on this essay at Lambda the Ultimate Guido van Rossum, designer of Python, discusses Weak and Strong Typing. Added 9 Feb 2003. Tim Bray, author of XML spec, on Future Language Fermentation. Discuss (2 responses) (Join / Login first) Edit   permalink: #

PHP is a love-hate relationship

Just visited loudthinking.com and David Hansson (who believe it or not appears to be a PHP supporter) wrote: > Specifically, PHP is sorely lacking in mature and widely applied > MVC frameworks, persistence abstractions, IDEs, testing suites, > and enterprise solutions. Frankly I come from the neanderthal era of computing, before the world-wide web, and I can say that many of the modern java practices are just a preference or methodology, and not a collection of universal "best-practices". 1. There exist MVC frameworks for PHP such as Phrame. This concept is nice, but i don't use MVC - it's just one way of doing things advocated by Smalltalk consultants turned Java advocates. There are many effective ways to control and manage your data and user interface. The key thing is to separate business logic and presentation (corrected 12 Dec 2002). 2. Persistence abstractions are also superficially nice, and I have implemented them in C and C++ using my own database schema and also using MFC serialization methods. Looking back at my past experience, they were a waste of time because of the overhead of the mapping layer (whether in C or PHP), and remapping when the data dictionary changed. However because virtually every modern OOP book discusses it, it looks really cool. The biggest headaches with persistent abstractions are (1) most dataset manipulation tasks are best done using SQL, not in objects and (2) problems with serialization and data migration (see Martin Fowler's interview). When you upgrade your objects, the serialization breaks. With PHP arrays (or Java Dictionary's) retrieved from an SQL statement, we don't have such issues. There are still some cases when persistent objects are better. One example is when you have a low-level datastore such as sleepycat's BDB, where PHP (or Java) objects provide a richer interface than the primitive database. 3. IDE's. I totally agree here. My two main gripes are (1) everytime i switch PHP versions (which is often as I have to test my PHP software on different PHP versions), I have to switch the Debugger/Zend Optimizer/etc, and (2) that refactoring tools are pretty poor in the PHP world. Most of the time, I just use homesite's regular expression replace, and CVS to undo any mistakes :-( 4. Testing suites. If you mean formal methods such as JUnit, then PEAR's PHPUnit is pretty good. 5. Enterprise solutions. I agree that PHP cannot be used for every part of an web-based enterprise solution. But for any type of coding that does not involve low-level work or intensive database processing, it's pretty good. In general, we find that we can use PHP for about 60-70% of our enterprise work. Our staff would have preferred to code 100% of our web-applications in PHP (it's so beautifully easy), but some things cannot be done in a 4GL. Discuss (3 responses) (Join / Login first) Edit   permalink: #

Reflections of the Day: Generalizations

Generalizations are generally wrong -- Butler Lampson Last Monday i gave a staff training on software methodologies. During the training, i asked them to draw a diagram describing "Extreme Programming". Everyone drew the requirements, design, development, testing, etc. But not many drew in their diagram the unique characteristics that differentiate extreme programming from other other methodologies (we also covered waterfall and spiral models). In fact, if you had seen what some had drawn, it would have looked like the classic waterfall model! I pointed out after looking at their diagrams that to be successful you have to identify the unique characteristics of the problem, and find solutions that specificly address those unique differences. If no project is treated the same, work becomes more fun. You may generalize first, but the truth is in the details. [Nowadays i occasionally post quotations at php.weblogs.com. I thought it would be fun to comment on them a few days after I post them, to see how it matches up with your opinions. I don't consider my opinions the best - it's just how i see things. Hope you find it interesting.] Discuss (Join / Login first) Edit   permalink: #

Does Perl has greater expressive potential than PHP?

BDKR posed this question in response to Thoughts on PHP. I do wonder about the simplicity or complexity of PHP. If it's simple, is it too simple, and as a result, how limiting of a factor is it? Should a language be expressive or merely a utility? Now where PHP compares to Perl, it seems that both are just as flexible. However, can we say that Perl has a greater expressive potential than PHP as a result of "pruning the core language"? Perhaps they've seen where Perl has gone and decided on a different course. What do you think? To me, expressive potential has nothing to do with the language and everything about who you are. Some people prefer to write stories and essays with fountain pens because they like to look at beautiful hand-writing. Others still use type-writers because they don't want to learn any other way. Some writers use Apple, others Dell. But the words that you write are your own, not the type-writer's, nor the computer's. Of course the programming language you use affects the way you do things. Certain designs become easy to apply, and others become harder. Perl and PHP do emphasise different things, but I do not think that they limit your expressive potential. You just make different compromises that suit your personality. In Perl, you use =~ / / for regular expressions, In PHP you use preg_match( ) or preg_replace( ). With PHP, you're Tom Hanks, straight-forward, honest and sensible. Perl is like Alan Alda, intellectual, strong and definitely a bit crazy (remember MASH!). Discuss (Join / Login first) Edit   permalink: #

Some Thoughts on PHP

I just received this interesting email from Derek Comartin. I thought I would share his message and my response with you. John, I have been reading PHPEverywhere ever since I found it about a year back. I love reading it and your thoughts about the latest tech. Just recently I started using ADOdb for a project and love it. I am not a huge fan of PEAR's DB or PEAR in general (although I am using its SOAP package). Anyways, the real point of my e-mail is that I am really getting frustrated with PHP. I have been developing applications with PHP for 3 years now, mainly developing intranet and CMS the like. My problem is that it seems like PHP is close to what I want it todo, but not quite. I would like to hear your thoughts and the thoughts of other php developers (people that use it everyday). Let me say first of all I have absolutely no influence on the direction of PHP. I do not have CVS access to the PHP source code, and wouldn't have the time to contribute even if I wanted to. However some people might be interested in my opinion. To me PHP is a pragmatic language. It is not based on any formal theory nor specification. It grew the same way as Perl, due to demand from skilled hackers. However Rasmus Lerdorf and company seem to be more keen on pruning the core language than the Perl gods and keeping it really simple. For example, to me the lack of the more advanced OOP features is a blessing. I was never impressed with C++ and its complexity, although i coded in it nearly every working day for 5 years. (BTW, sorry if this is going to get long.. my real goal here is maybe you can post this on your site and hope to get some feedback in your comments section). 1) Templates Almost everyone agrees that mixing HTML and PHP is a bad idea. Personally I have developed my own template library that is simple and thin. Since PHP is geared toward web applications why in gods name don't they develop a SIMPLE template library built-into PHP. I cannot understand for 1 minute the point of smarty? I do think smarty is a very well designed library but I don't understand the point of making a library that introduces more business logic. Isn't the point of a template to seperate buisiness logic from UI? Who is coming up with this stuff? The development of templates was an attempt to fight against complexity. This is good. Yes PHP is a template-based language, but that doesn't mean that good techniques preclude more advanced strategies for separating code,data and presentation. For very big scripts, we split code into functions and into several include files. We store data in an RDBMS for better management. So for very big web pages, it makes sense to split it into multiple files, and split presentation to template files too. Then we have a N-layer architecture of: DATABASE   | PHP BUSINESS LOGIC   | PRESENTATION LAYER TEMPLATES   | PHP ENGINE   | APACHE From a theoretical point of view, perhaps Smarty has gone too far. Smarty does not merely deal with presentation issues, but has a full-blown programming language built-in. There is always the temptation to add functionality to something that began as a simple project. In some ways, Smarty compiled code is obfusticated PHP :-) However from a commercial perspective, Smarty is cool. I can release a Zend encoded PHP product, and provide customization features by basing my user interface on Smarty, which the client can modify and even script without touching my compiled code. So it really depends on your perspective. One man's Toyota is another man's Rolls Royce. And some people refuse to learn to drive and will never see the point of using Smarty. (BTW: I have been wondering if someone has written a template library that is something like this: If you had a template that had a <form id="myForm"> tag, then in your PHP you could modify tag properties etc... im not sure if ASP.net does something like this.. so your code would be: $tmpl->myForm->setAction('myform.cgi'); $tmpl->myForm->setMethod('POST'); Kinda like how Javascript can minipulate objects, you could do with any HTML tag.. ie: forms, tables, yadda yadda There are several libraries that do this already if you search on hotscripts.com. including (wink-wink) my company's product, phpLens which goes one step further in providing graphical ui designers/wizards. And my point of view is a little bit more radical - doing this without graphical aids is a step backward. For example this doesn't excite me: $tmpl->myForm->DrawInput('text',$size=32,$maxlength=64); because you lose the benefits of a graphical tool like dreamweaver that recognizes input tags for the obscure benefit of coding everything as objects. Half the power of MS.NET is the graphical tools of Visual Studio.NET. Note that I had to use $size and $maxlength before you could even guess what the parameters of the function were. The beauty of HTML is that it is self-documenting. 2) Database Thanks to you I use a great database abstraction. But why on earth isnt there abstraction built-in? I still like having functions for a specific API, but there should be db abstraction on top of that built in for speed. Anyways these are just my random thoughts... I love PHP but there are too many things that are beginning to piss me off (did I also mention nonsense like how functions like explode, strstr have there haystack/needle arguments in different orders? Shouldnt this be common across all functions.... ah the little things.) What are your thoughts? PHP was developed to meet a need. PHP has grown organically, not because it was sponsored by big companies like Sun or IBM (eg java). People originally used PHP to create web-sites where the database was probably known; in contrast, database abstraction is only required if you are creating web applications that can be installed in different environments. I rarely do web-sites, most of my PHP work is for web-applications that run on Intranets/Extranets. That's how ADOdb came about. That's how PHP came about - to meet an immediate need. So as PHP has matured, so has the needs become more sophisticated. Some people will always want PHP to do more and grow in more powerful ways (data abstraction, 100% OOP, etc), but as PHP growth is organic, you have to wait till someone faces the problem and comes up with the solution and post it to the net. This laissez-faire approach works because the number of PHP developers has reached a critical mass (perhaps when PHP4 was released). Of course some things could have been better planned from the beginning of PHP, but no one would have started work till the last bit of the plan was nailed solid, which would have taken too long. Instead Zeev and Andi (and Rasmus?) are still very conservative about the core language, and they let you and me build whatever we like without comment. PHP-Nuke and Post-Nuke might never have been developed if Rasmus had blessed Midgard as the way to go. Of course there are people who say that the Nuke forks cannot compare to Zope or Soup or Whatever, or that the PHP api is not as nice as Java's or Python's -- but that misses the point. Millions of people are using and benefiting from the free software when programming in Java or Python or C# would have been too difficult. And this gives impetus for people to create better software. -- John Lim Discuss (6 responses) (Join / Login first) Edit   permalink: #

Is Design Dead?

For many that come briefly into contact with Extreme Programming, it seems that XP calls for the death of software design. Not just is much design activity ridiculed as "Big Up Front Design", but such design techniques as the UML, flexible frameworks, and even patterns are de-emphasized or downright ignored. In fact XP involves a lot of design, but does it in a different way than established software processes. XP has rejuvenated the notion of evolutionary design with practices that allow evolution to become a viable design strategy. It also provides new challenges and skills as designers need to learn how to do a simple design, how to use refactoring to keep a design clean, and how to use patterns in an evolutionary style. -- Martin Fowler A deep and subtle article with many nuances. Highly recommended, but you might want to reread it several times in your career to fully benefit from it, as many of the things discussed mean different things to you when you are a programmer, software architect, manager or tester (all hats which I have worn). I'm going to search for books written by Martin Fowler. He's a good guy who uses common sense to navigate his way around the jargon and myths surrounding software design and engineering. Slashdot book review: Questioning XP. Discuss (1 response) (Join / Login first) Edit   permalink: #

Is EJB Always Necessary?

Like every technology, Enterprise JavaBeans have many good and bad points and must be used with caution. I feel that great thought must be used before deciding to use EJB in a project. Unfortunately at the moment EJBs are the default choice - probably because they get so much media attention, not because they're technically the best fit for the project. The intention here is not to discredit EJB but to present experiences and share alternatives and tips. -- Patrick Lightbody and Mike Cannon-Brookes I am studying Enterprise JavaBeans nowadays because in my PHP work, I'm beginning to face enterprise scale problems. After reading this article, and reading responses of other Java experts such as Gordon Sefchik, perhaps it was a waste of time. They say EJB is scalable, but I find you need tons of good hardware to achieve good performance (not surprising given the business of Sun). As others have noted, the success of Google, Yahoo, etc has demonstrated that commodity servers using simpler programming models can work well. Now people using Java are beginning to see the light too. Discuss (Join / Login first) Edit   permalink: #

Software Engineering Practices for Large-Scale PHP Projects

An excellent list of best practices that Scott Johnson presented at PHPCon 2002 that i would highly recommend following. The only problem is that these slides do not have notes, so you have to fill in the assumptions yourself, which might not be easy for novices. Discuss (1 response) (Join / Login first) Edit   permalink: #

Making the case for PHP at Yahoo

Running a high-performance dynamic website is a daunting task. The short development cycles needed to stay ahead of the competition demand a web-centric scripting language that is easy to maintain and update. We'll explore a case study of one company (Yahoo!) that is making the transition to PHP from a proprietary server-side page language written in C/C++. -- Michael J. Radwin (Apache lead, Yahoo) This is an excellent article that shows Yahoo seriously evaluating several different technologies, including ASP, Cold Fusion, JSP and Perl, with PHP winning out. It looks as if most future development of Yahoo will be in PHP. PS: Jeremy Zawodny, MySQL guru at Yahoo is blogging PHPCon 2002. PHPCon 2002 review by Kawika Ohumukini. Discuss (Join / Login first) Edit   permalink: #

Staged Event-Driven Architecture for Internet Services (pdf)

As a more concrete example, Figure 1 shows the load on the U.S. Geological Survey Pasadena Field Office Web site after a large earthquake hit Southern California in October 1999. The load on the site increased almost 3 orders of magnitude over a period of just 10 minutes, causing the Web server’s network link to saturate and its disk log to fill up [142]. Note that this figure shows only the number of requests that were successfully logged by the server; due to the overload, it is likely that an even larger number of requests were present but not recorded. During the load spike, the system administrator who was responsible for the Web site was unable to login to the system remotely to clear up the disk log, and had to physically reboot the machine in order to regain control. The most common approach to dealing with heavy load is to overprovision resources. In the case of a Web site, the administrators simply buy enough Web server machines to handle the peak load that the site could experience, and load balance across them. However, overprovisioning is infeasible when the ratio of peak to average load is very high; it is not practical to purchase 100 or 1000 times the number of machines needed to support the average load case. -- Matt Walsh Discuss (Join / Login first) Edit   permalink: #

Enterprise PHP

If you are too popular, jealous people are always saying behind your back that you are a compulsive schmoozer, or you are insecure, or insincere. Well PHP is an Open Source language that is widely popular on the web. However because PHP so popular in shared hosting environments, many people have an impression that PHP is only for small scale web-sites. This is patently untrue, and PHP is in use in many large scale web sites such as Yahoo Finance and for the creation of large web applications such as IMP. This article is an attempt to readdress the balance and show how PHP is used in the enterprise. Discuss (Join / Login first) Edit   permalink: #

Six Common Enterprise Programming Mistakes

Instead of giving you tips to use in your programming (at least directly), I want to look at some common mistakes made in enterprise programming. And instead of focusing on what to do, I want to look at what you should not do. Most programmers take books like mine and add in the good things, but they leave their mistakes in the very same programs! So I'll touch on several common errors I see in enterprise programming, and then briefly mention how to avoid those mistakes. -- Brett McLaughlin One of the best ways to improve your software design skills is to have a look at what experts in other languages are thinking, and reflect that back in your PHP code. You learn the most when you work out for yourself whether or not a good idea in one language applies to PHP. Good stuff. Discuss (Join / Login first) Edit   permalink: #

Lessons from Giant-Scale Services (PDF)

Describes experiences in managing large scale web farms, specificly AOL, Inktomi, Geocities and what appears to be Yahoo Mail. Very interesting from a geeky point-of-view, but the double-column format is a pain to read. Print it out. Discuss (Join / Login first) Edit   permalink: #

A unified theory of software evolution

Meir Lehman has been studying the life cycles of computer programs since he was a researcher at IBM 30 years ago. One of these days he's going to get it all figured out. It's interesting to read this because I have been struggling with my own legacy code issues. It's very easy to write PHP code. You can pluck variable names from the sky and plant them in your code anywhere with no discipline. Well my free-wheeling style came back and bit me. It's very dangerous to declare new variables such as $i and $j in a thousand line function because such variables are so common. Of course I accidentally re-initialized a variable that was already in use 500 lines above. The lesson here is one that is as old as "software tools". Two unix gurus in the 70's wrote one of the greatest software engineering books of all time: Software Tools by Kernighan and Plauger. Unlike other software engineering books that focus on process and procedures, this book concentrates on explaining how to write good code: Write modular code composed of small functions that perform specific operations, and link them together as a toolkit to perform more powerful tasks. Still very relevant today. -- John Discuss (2 responses) (Join / Login first) Edit   permalink: #

The Myth of Open Source Security

Recently, I had to update the Linux servers I am maintaining because of ssh and PHP security problems. It makes me think about the people who are always complaining about Windows being insecure. It's true that Windows has a bad record. But everyone else (except BSD perhaps) has a bad record too. John Viega, author of GNU Mailman explains why... So I am glad that PHP is currently going through a security audit by some OpenBSD volunteers. It's interesting to see how the PHP community reacts. Some people don't like it. Perhaps they feel the OpenBSD guys will kidnap the project or make them look bad. Zeev shows his class by welcoming them with open arms. Discuss (Join / Login first) Edit   permalink: #

Microsoft Guru: Stamp out HTTP

[HTTP] works for small transactions asking for Web pages, but when Web services start running transactions that take some time to complete over the protocol, the model fails. "If it takes three minutes for a response, it is not really HTTP any more," Box said. The problem, said Box, is that the intermediaries--that is, the companies that own the routers and cables between the client and server--will not allow single transactions that take this long. -- Don Box Interesting point, but this alarmist attitude only makes ignorant chickens squack in fear. The solution already exists. It's called message queuing. Microsoft even has a queuing product called MSMQ. The algorithm: For every long-running RPC call coming from a client, you store the transaction on the server in a queue and disconnect. For each transaction in the queue, you also store a client callback address. When the transaction is popped off the queue and carried out, you send the response back to the client callback address. -- John Discuss (1 response) (Join / Login first) Edit   permalink: #

PHP and ASP benchmarked again

This article is in French, but luckily the language of numbers and programming is universal. There were 2 benchmarks, one for generating a browse list of products using and SQL query and merging the results into a template, and another benchmark, drilling down to view a single product. ASP/IIS won the first, while PHP/Apache won the second. This may sound inconclusive, but I think that the testers were good ASP programmers, but PHP novices. The first benchmark tests string manipulation extensively, but they used the worst way of manipulating strings, the ereg functions. Using str_replace() is the equivalent of the VB Replace() function and is much faster. I would have loved to see the ASP programmers use regular expressions in their code - it would have been slow as molasses! PHP code tends to have slightly faster database access than ASP because there is no COM/OLEDB overhead. That is why I suspect PHP managed to move ahead in the 2nd benchmark despite the regular expression handicap. The lesson here should be not to say that the benchmarks were fixed, but that the PHP education process could be improved: the str_replace() documentation does say that this is faster than ereg_replace(), but it is the wrong place for the note - it should be placed in the ereg_replace() and preg_replace() sections where they can do most good. Anyone with CVS access to the PHP docs reading this? Lastly, if they had put ob_start() at the beginning of the PHP code, we would have got another 2-5% performance improvement. Output buffering maximizes network throughput, and is the default for ASP on IIS5. PostgreSQL, MySQL and MS SQL Server are also compared in this benchmark, with MySQL the winner for speed, and PostgreSQL wins the scalability award! Discuss (Join / Login first) Edit   permalink: #

Microsoft Renames Hailstorm to MyServices

From: "Dann Sheridan" <arkangel9@y...> Date: Sat Dec 15, 2001 1:53 pm Subject: Microsoft Publishes Hailstorm Schemas All: Looks like the lid is off and has been off for some time. HailStorm is now called "MyServices." This was probably the quietest launch from Microsoft I've ever seen, which should give you an indication of how strategic they view MyServices. Below is a list of URLs that will take you to the long awaited HailStorm schemas. There is a lot to digest. The XMI is published here: http://www.microsoft.com/MSPress/books/5800.asp MyServices home page: http://www.microsoft.com/myservices/ Schemas: http://schemas.microsoft.com/hs/2001/10/hscommon.xsd (core) http://schemas.microsoft.com/hs/2001/10/hsdl.xsd (core: security) http://schemas.microsoft.com/hs/2001/10/hsinfra.xsd (core: security) http://schemas.microsoft.com/hs/2001/10/myCalendar.xsd http://schemas.microsoft.com/hs/2001/10/myInbox.xsd http://schemas.microsoft.com/hs/2001/10/myApplicationSettings.xsd (core: software licensing) http://schemas.microsoft.com/hs/2001/10/myContacts.xsd http://schemas.microsoft.com/hs/2001/10/myDevices.xsd http://schemas.microsoft.com/hs/2001/10/myDocuments.xsd http://schemas.microsoft.com/hs/2001/10/myFavoriteWebSites.xsd http://schemas.microsoft.com/hs/2001/10/myLocation.xsd http://schemas.microsoft.com/hs/2001/10/myServices.xsd http://schemas.microsoft.com/hs/2001/10/myCategories.xsd http://schemas.microsoft.com/hs/2001/10/myAlerts.xsd http://schemas.microsoft.com/hs/2001/10/myPresence.xsd http://schemas.microsoft.com/hs/2001/10/myWallet.xsd http://schemas.microsoft.com/hs/2001/10/myProfile.xsd This is all quite exciting. Microsoft is now well positioned to dominate the next generation of the Internet: web services. Or, at the very least, Microsoft has validated web services as the future of software development. From what I have seen in these schemas, all rolled up, it looks like the gap between developers and revenue generation is closing. Enjoy! -Dann John: Trying to make sense of all this? Read these in-depth interviews of Mark Lucovsky and Shaun Pierce, Hailstorm architects. It discusses many of the underlying design issues behind Hailstorm. Discuss (1 response) (Join / Login first) Edit   permalink: #

Designing Fast Software Architectures in PHP

The recent benchmarking spree of PhpLib, ADODB, PEAR DB, Metabase and Native MySQL has brought more questions up. Here's my opinion. 1. Are the benchmarks reasonable? 2. Why is there such a big difference in performance? 1. Reasonable Benchmarks Every benchmark is a simplification. What is important is whether the simplification does some useful measurement. What we are measuring is how efficient the class libraries are in performing a select statement retrieving 82 rows of data containing 4 columns from the database. This seems to be a typical task on a Web site and so we pass the reasonableness criteria for scripting dynamic HTML pages. 2. Differences in Performance The code is also simple enough that programmers better than me can quickly spot errors in my implementation; I have revised the code three times because of feedback. However the speed improvements have only been marginal (say 10%) and not big changes. This suggests that architectural differences in the libraries are the main cause of the speed difference. Also a quick look at the code and you will realize that the speed of the class library in this benchmark is proportional to the lines of code and functions executed - perhaps code bloat is a cause for some slow-downs. So a discussion of how to design fast software architectures is useful. Designing Fast Architectures When designing a class library that is used by many people, it is important to divide your code between core and peripheral. Core code is used everywhere, so it is best to reduce the feature-set of core code, and move these nice features into the peripheral section. This makes it easy to tune the core code for speed because the code is so simple. Some of the class libraries made the mistake of not dividing the code up into must-haves (core) and nice-to-haves (peripheral), so when they added features, the core code became "polluted" with slow non-essential code that is rarely used. Also avoid overly-complex designs with a lot of message passing and clever object hierarchies. They are great for impressing people but rarely run fast in real life. PhpLib is to be admired in that sense - it's fast and is not pretentious. The next thing to ask ourselves is what is the natural data size when dealing with database tables when using a 4GL like PHP. No it is not the field or column, it is the row or 1-dimensional array. Databases are tuned for sending records for that reason. A PHP class library that tries to operate purely at the field level is going against "nature" in that sense. Thus it comes as no surprise that these libraries that are prepared to struggle uphill against "nature" are extremely slow. It would have been better to have a fast core that operated at the record level, then have peripheral functions that handle fields and other recordset details specially (if that level of flexibility is required); this is ADODB's design. See http://phplens.com/lens/adodb/ for the benchmarks of database abstraction libraries. I am preparing an extended version of this article for my forthcoming PHP Handbook to be published by Sams Publishing, out in 2002. Discuss (2 responses) (Join / Login first) Edit   permalink: #

Richard Gabriel: Worse is Better

A classic of computing written in 1989. Richard Gabriel looks back on Worse is Better. Discuss (Join / Login first) Edit   permalink: #

Good Software Takes Ten Years. Get Used To It

Don't get too hung up on your version 1 and don't think, for a minute, that you have any hope of reaching large markets with your first version. Good software, like wine, takes time. -- Joel on Software Discuss (Join / Login first) Edit   permalink: #

First Rule of Usability? Don't Listen to Users

In past years, the greatest usability barrier was the preponderance of cool design. Most projects were ruled by usability opponents who preferred complexity over simplicity. As a result, billions of dollars were wasted on flashy designs that were difficult to use. -- Jakob Nielsen Discuss (Join / Login first) Edit   permalink: #

Exploiting Common Vulnerabilities in PHP Applications

Shaun Clowes gives an excellent overview of PHP security gotchas. He also gives a set of security recommendations at the end of the essay. A must read. Discuss (Join / Login first) Edit   permalink: #

Writing Smart Web-based Forms

Forms are used frequently in dynamic web applications. As a consequence, almost all web programmers have to deal with form data validation at some point in their careers. The technique described in this article will help you help the user submit correct information. Discuss (Join / Login first) Edit   permalink: #

PHP Case Study: From Hot Concept to Hot Site in Eight Days

HotOrNot.com evolved from an idea into one of the biggest sites on the Web in less than two months. Such rapid growth meant the site had to scale quickly, especially in the first eight days. Discuss (Join / Login first) Edit   permalink: #

Building Web Applications with C#

The ASP+ page framework, known as Web Forms, is the new runtime programming model used to create dynamic Web pages. With this model, legacy ASP pages are still supported, allowing developers to ease into the transition to ASP+. Web Forms provide a number of benefits to the Web developer, including powerful Web controls and validation controls, ViewState management, and enhanced performance delivering requests. Discuss (Join / Login first) Edit   permalink: #

PHP Coding Guide

The experience of many projects leads to the conclusion that using coding standards makes the project go smoother. Are standards necessary for success? Of course not. But they help, and we need all the help we can get! Be honest, most arguments against a particular standard come from the ego. Few decisions in a reasonable standard really can be said to be technically deficient, just matters of taste. So be flexible, control the ego a bit, and remember any project is fundamentally a team effort. -- by Fredrik Kristiansen. The fun part is too see how many of the coding standards I violate in my code. I had at least 6 violations. How about you? Seriously though, 99% of the ideas in the document are good. Just remember that the above is not really a standard, but a guide. Hats off to Frederik for taking the effort. Discuss (2 responses) (Join / Login first) Edit   permalink: #

Unix Web Architectures

Above is a nice basic overview on designing a Web application by Samuli Kärkkäinen. For another view, read the ArsDigita article. Discuss (Join / Login first) Edit   permalink: #

Web Services with PHP using XML-RPC

One of the most important ideas in creating scalable Web sites is to decompose our programs into more modular Web services. Our Web pages will no longer read a database, process the business logic and spew out HTML. Instead we will subdivide our pages into multiple sections, and render each section by calling different Web services. In this way we spread the CPU load across multiple servers. This article shows you how to implement these Web services using XML-RPC. Discuss (Join / Login first) Edit   permalink: #

ArsDigita: Building Scalable Web Services

"Web services are usually built with loose specs, to tight deadlines, by people who aren't the world's best educated or most experienced programmers. Once online, Web applications are subject to severe stresses. Operations get aborted due to network failure. So if your transaction processing house isn't in order, your database will become corrupt. You might get dozens of simultaneous users in a second. So if your concurrency control house isn't in order, your database will become corrupt." An excellent tutorial on building scalable web systems on Unix. Discuss (Join / Login first) Edit   permalink: #

PalsLib Linux Database Library

This site is a compilation of the best free online readings about relational databases on Linux. Discuss (Join / Login first) Edit   permalink: #

JSP Reinvents the Wheel

An interesting article about The Problems with Java Server Pages. Turns out that Java is unsuitable for the dynamic prototyping required for Web pages, so people are reinventing the wheel by building Scripting Engines (similar in functionality to PHP) on top of JSP. My solution - junk JSP, stick to something simpler such as PHP or ASP. Read the responses to the article. Discuss (Join / Login first) Edit   permalink: #