I’ve already posted on this in the Chemistry Direct group and Chem Direct team have responded with some suggestions; however the other side of the improvement gains (and the big ones) I think could come from optimising the way AEP itself is handling our server resource.

We have AEP9.1HF2 running on RHEL5.10 (Linux version 2.6.18-371.3.1.el5), a quad core server (single CPU) recently upgraded from a Dual core. We have noticed that protocols when running only ever utilise 1 core, even if the other cores are not being used.

We are running a computationally intensive Pipeline Pilot (AEP9.1) protocol which we would like to speed up. We have (and are) optimising the protocols themselves, but we would also like to make full use of the server power (and potentially add more cores / CPUs to increase performance).

The majority of the time (90%) is spent running Custom FlexMatch, and Markush searches against a Chemistry direct database (Direct 9 / Oracle 11 Standard Edition) on the same server (the Chemistry Direct team have already posted some good suggestion that we've implemented at https://community.accelrys.com/message/19738#19738)

A small reference test of the protocol on our RHEL Dual core Xeon server, 8Gb RAM took ~2 minutes. During this time the server resources peak at ~50% with 2 cores being utilised to ~50%. Memory is good. Thus we can have 2 separate jobs running this protocol concurrently without much degradation (on a dual core).

We hoped that upgrading to a Quad core processor would increase the speed of each protocol, e.g drop from 2 to 1 minute. However instead the protocol takes the same amount of time to run, but on the 4 core server we can have 4 jobs running it in parallel without performance degradation. See below, CPU usage with 1 job running

10:24:33 AM       CPU     %user     %nice   %system   %iowait   %steal     %idle

10:24:34 AM       all     16.00     7.25     3.00     0.00     0.00     73.75

10:24:34 AM        0     34.00     0.00     7.00     0.00     0.00     59.00

10:24:34 AM         1     28.00     29.00     3.00     0.00     0.00     40.00

10:24:34 AM         2     0.00     0.00     0.00     0.00     0.00   100.00

10:24:34 AM         3    2.02     0.00     1.01     0.00     0.00     96.97

And the breakdown by application

PID USER     PR NI VIRT RES SHR S %CPU %MEM   TIME+ COMMAND

8860 oracle11 16   0 2437m 50m 45m S 41.9 0.6   0:11.88 oracle

8865 oracle11 16   0 287m 42m 26m S 40.2 0.5   0:10.28 extproc

4766 aep91pro 25 10 800m 416m 38m S 17.0 5.3   0:15.13 scisvr

15214 aep91pro 20   0 1102m 203m 17m S 1.7 2.6   0:25.19 scihttpd

(note both screenshots taken are slightly different times with different tools, hence the difference in absolute numbers).

From what I've seen it looks like when a protocol starts AEP takes ~50% of 1 core and Oracle/Direct ~50% of another as well.

What we really want is that if that only 1 protocol is running, AEP uses all 4 cores to maximum effect, in effect reducing the processing time from 2 minutes to 30 seconds.

In the admin portal / server config our server is set to use the maximum number of cores (setting is left blank)

How can we force / encourage AEP to use all 4 cores at once to speed up the running of the protocol? Due to the nature of pipeline pilot work I would have thought this would be default behaviour.

Thanks

Joe