Shaz posed an interesting question in my recent post on caching:

My website is heavily personalized once a user logs in. I've been struggling trying to find ways to speed things up, because almost every kind of caching that you hear about is for static pages or dynamic pages that can be cached because they are the same for everyone.

The best that I've been able to come up with is storing things in the session when the user logs in. That way I don't have to do database calls on every page for common data, because the common data is always available in the session. However, this doesn't work for everything, because some data even changes after they've logged in. This is the best I can come up with though. Anyone have any comments?

Dear Shaz,

It is still possible to perform clever caching techniques that maximize performance. Off my head, here are a few:

• As you mentioned, storing in session variables is a good way of caching personalized content, provided the variables are used on every page, and it's not too much data.

• In the ideal case, try to keep personalised info in one area of the screen. Then you can cache the HTML, instead of the personalisation variables. One good way of displaying this information is in an IFRAME.

• For refreshing stale data, store the userid and an update counter in a database table. When the person logs in, read the update counter. Create triggers that increment the counter whenever data changes. The triggers can be in the database if you are using PostgreSQL, Oracle, or similar database, or implemented in your PHP code.

  Every time you access a page, check the update counter for that user and compare it with the value you are storing in a session variable. If it changes, reload all personalisation variables again. This works best if the cost of reading one update counter table is much less than getting the actual data.

• Another trick is to cache the data on the browser. Create a javascript file that is actually a PHP script, and pass it to the browser when the user logs in, with the appropriate HTTP caching headers. The javascript file would contain personalisation stuff like:

  theUser='John Lim';
  theFavoriteIcon = 'iconJL321.gif';
  

  Then in your HTML files, do something like this to output the personalised info.

  <script> document.write(theUser) </script>
  

  This technique allows you to cache the entire HTML page (on the server), and cache the personalisation info too (on the client) as javascript. To me this is simple and beautiful.

A close study of Oddpost and Gmail should give you lots more ideas too. However using such clever techniques probably require a total rewrite. If any reader has more good ideas, please post it in the comments.

Update: Michael Radwin discusses HTTP caching in OSCON 2004. Points out that cookies do not need to be transmited for static images, and this requires separate domain. You really have to use a proxy software like TCP Trace to monitor HTTP behaviour.