Debugging techniques in ESP.
1. Using print statements: If user is using SPLASH then it is easy to output intermediate values using print statements.
print(‘Intermediate calculation ‘,to_string(val1));
will display the value of val1 in the stdstreams.log file. By introducing simple print statements in the SPLASH script users can check the values to see if the calculation is correct.The print statements come up in the log file only after the project is stopped.
2. Event stream processor (ESP) comes with a debugger which can be used to debug a running project. The ESP debugger can be invoked using Studio as well as using esp_client. By using breakpoints on the output or input of a particular stream, users can view the events that went in to the stream as well as the events that come out of the stream. Help on this feature is available under
http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01688.0500/doc/html/skh1303825703022.html. Attaching a debugger can slow down the system.
3. Error streams are another good technique to find out the data that caused a particular error. For example if there is a floating point exception error or some data gets rolled back and there is a error message in your esp_server.log. To find out which data caused this error a user can introduce a error stream and check which data caused the error message. More help on this is available under http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01613.0500/doc/html/lar1320437138260.html. When there are too many errors there is performance degradation when using error streams.
4. Dumping intermediate values using the output statement is also a good technique to use when debugging complicated SPLASH code. Make the output of the flex operator as a stream and modify the schema to match your output statement. As streams don’t need key fields it is easy to see the results of any type of output.
5. esp_subscribe command can be used to subscribe to any input/output stream/window and check the data that is going in to the next stream in the data flow. More help on this is available under http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01688.0500/doc/html/cgo1317756844781.html. The advantage of using esp_subscribe is it shows the opcode of the data that is passing to the next stream along with the data whereas the Studio stream viewer does not show the opcodes. Also the Studio stream viewer output updates row values based on the opcodes, so it is not possible to see the output history overtime. For performance reasons the stream viewer has a limited number of rows that show in the output whereas the esp_subscibe command does not have these limitations and can be easily redirected to a file so that the user can keep the output in a file and inspect it anytime.
6. Event cache data, dictionary data, vectors, and global variable values of a running project can be viewed using esp_client command.
esp_client –p localhost:9786/default/testeventcache
ex `var` `S3` `stats` where stats is the event cache in the stream S3. More help on this is available under http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01688.0500/doc/html/kra1305745129271.html. Currently there is no way to see this using Studio.
7. Performance metrics can be collected using metadata streams. The Streams_Monitor provides performance metrics on every stream in the running project and the Connectors provide metrics on the input output adapters in the project. More help on the feature is available under http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01611.0500/doc/html/skh1300802200896.html. The metadata streams can be viewed in Studio as well as accessed via the esp_subscribe command line command.
8. To figure out the memory consumption of each stream a user can increase the debug-level of the project to a 7 and run the project with the test data and after completion of the run when the project is stopped
the esp_server.log file will show the memory usage on a per stream basis something like this.
2012-03-13 13:06:33.253 | 8712 | container | [SP-6-131039] (107.544) sp(9652) CompiledAggregateStream(BidBook): Collecting statistics (this could take awhile).
2012-03-13 13:06:33.253 | 8712 | container | [SP-6-124001] (107.544) sp(9652) CompiledAggregateStream(BidBook)::Memory usage: 359,600,246 bytes in aggregation index.
2012-03-13 13:06:42.175 | 8712 | container | [SP-6-131039] (116.466) sp(9652) CompiledAggregateStream(BidBook): Collecting statistics (this could take awhile).
2012-03-13 13:06:42.175 | 8712 | container | [SP-6-131040] (116.466) sp(9652) CompiledAggregateStream(BidBook): Memory usage: 327 bytes in 1 records
The loglevel of a project can be modified at run time also by using the esp_client loglevel command.
9. To monitor the performance of the project ESP has a Studio monitor utility and esp_monitor command line utility. This utility provides insight on which stream/query causes performance degradation. The Studio monitor is a pictorial representation of the project performance whereas esp_monitor is a command line utility showing the no of messages each stream processes. More help on this is available under http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01611.0500/doc/html/tbi1308861286789.html
10. Another way to see the events propagating is by using the event tracer in Studio. Users can click on event tracer, select the project, use the Manual Input to publish data to their running project, then use the event tracer to check the data propagating through individual streams by double clicking on the nodes in the event tracer and watching the console output which shows the opcode and data that propagated through each stream. More help on this is available under