SOAP connector result transfers

Hi,

I’m hoping someone may have seen this before and know the answer…

I’m seeing a strange thing when using the AEP SOAP connectors. Whenever I run a protocol I’ve written to call a pipeline pilot protocol via SOAP the longest part is the Start Job (upload data) and Get Results. The more results I have the longer it takes (as expected), however the time taken is exponential to the file size!

For example to return 10000 results generates a 1556Kb sdf file. This take over 3 minutes to return in Get results. However the actual SOAP service calculates the results in under 20 seconds (not dependant on file size)! So the transfer of data via SOAP is most of the time.

Uploading or downloading these files on the server from the client is pretty much instant, so it’s not a bandwidth / network issue as far as I can see.

The time it takes seems to increase exponentially relative to the file size for upload / download.  The table below shows doubling the data in / out of the SOAP service creates delays much longer than double the time.

Records

Start job (seconds)

return data (seconds)

File size (Kb) upload

File size (kb) download

1000

0.14

1.76

13

150

5000

1.56

37

66

750

10000

5.77

152

131

1500

20000

22.2

>600

263

3000

30000

50.1

>>600

4500

During this time the Apache server is taking up an entire CPU core. This doesn’t stop even after start job times out.

Has anyone seen this before? Any thoughts as to what I could try? I’ve attached a protocol showing the steps I can to connect to the SOAP service. You won’t be able to run it, but it gives an overview of what I’m trying to do.

Thanks

Joe