Caching in Opentaps / OFBiz

The Entity Engine of OFBiz and Opentaps has a very useful built-in caching capability. Caching improves overall performance of the application and substantially reduces database hits. OFBiz caches entities, properties files, freemarker templates, beanshell/groovy files, forms, screens and other xml and config files.

The entity definition for OFBiz has a field called ‘never-cache’. If this is set to true, caching of this entity will not be allowed. Automatic cache clearing is not done to improve efficiency and any attempt to use the cache methods on the entity results in an exception so that it is easier to find and eliminate where this is being done. Its value must be true or false and if not specified, it defaults to false. We can cache view entities as well.

It is strongly recommended to use find methods of GenericDelegator that have corresponding cache methods. For example – findByAndPrimaryKeyCache, findByAndCache, findByConditionCache and findByAllCache. These methods look for respective entities that are in caches and return the cached GenericValue objects. If entity is not configured to be cached, then a warning is thrown and the request is processed by querying the database.

All entity caches update automatically if there are any changes in the corresponding entities.

The Entity Engine cache uses the OFBiz UtilCache class to implement the actual cache. UtilCache provides a number of caching features -

- Limited or unlimited cache size,
- expiring cache entries after a configurable amount of time
- soft references so that the garbage collector can reclaim entries from large caches when more memory is needed
- If limited, removes elements with the LRU (Least Recently Used) algorithm
- Keeps track of when each element was loaded into the cache
- Using the expireTime can report whether a given element has expired
- Counts misses and hits

All these can be set on instantiation of UtilCache object by calling one of its constructors. There are two ways to configure UtilCache based caches -

1. Configure cache.properties file (as documented in the Core Configuration Guide here http://ofbiz.apache.org/docs/coreconfig.html#cache) for permanent changes.

2. Make temporary changes through the Cache Maintenance pages in the WebTools webapp. Those pages can also be used to view statistics, clear cache values, and perform other cache related maintenance.

Manual Cache Clearing -

Clearing all caches is a heavy process with a temporary performance hit and should be avoided as much as possible on production. It is recommended to clear specific caches only. For example – if updates to ftl files were merged on production codebase and if ftl files are cached, it is recommended to clear cache with name ‘template.ftl.general’ rather than everything.

Ftl Cache

Each cache has a name and we can clear caches based on their names -

Cache Names

Rather than clearing cache from webtools, we can automate it with a little additional coding.

Manual Cache Clearing Use Case –

We generate and store all SEO URLs in a properties file. The service generating these URLs runs every night and we want to clear the properties cache after this service is executed.

1. Inside UtilProperties.java, implement your cache clearing method -

public static void clearResourceCache(){
UtilCache.clearCache("properties.UtilPropertiesUrlCache");
resourceCache = new UtilCache("properties.UtilPropertiesResourceCache");
}

2. Inside your Events class, define a method that calls the above method -

public static String clearSeoCache(HttpServletRequest request, HttpServletResponse response){
UtilProperties.clearResourceCache();
return "success" ;
}

3. Define the request map in controller.xml -

<request-map uri="clearSeoCache">
<event type="java" path="com.xyz.UtilEvents" invoke="clearSeoCache"/>
<response name="success" type="view" value="main"/>
</request-map>

4. To avoid access of this from outside world, you can IP restrict the access at apache level -

<Location /xyz/control/clearSeoCache>
Order deny,allow
Deny from all
Allow from 110.218.194.16
Allow from 11.91.13.26
Allow from 127.0.0.1
</Location>

5. Create a shell script file called clear_cache.sh -

#!/bin/bash
wget -q http://localhost/xyz/control/clearSeoCache

6. Schedule a cron job that runs the script file every night at 4am -

0 4 * * * /home/opentaps/production/etc/deploy/clear_cache.sh

Distributed Cache Clearing -

If we have multiple clusters, we can clear caches of all servers using the above technique and add more wget requests in the clear_cache.sh shell script file as follows -

#!/bin/bash
wget -q http://cluster1/xyz/control/clearSeoCache
wget -q http://cluster2/xyz/control/clearSeoCache
wget -q http://cluster3/xyz/control/clearSeoCache
wget -q http://cluster4/xyz/control/clearSeoCache
….

Caching Drawbacks -

In our experience, Caches are prone to memory leaks and can also hog up the memory really quick depending on the records in the entities. Any custom defined entities have to be thought through well before deciding to cache them. You can run the Eclipse Memory Analyzer Tool (MAT) (http://www.eclipse.org/mat/) to check for leaks.

Command to generate the heap file for MAT-

/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/jmap -heap:format=b OFBIZ_PID
or
/usr/java/jdk1.6.0_14/bin/jmap -heap:format=b OFBIZ_PID