Using Timing Statistics to Optimize Tests

Would you want to make your tests run faster?

rflogs.io offers Timing Statistics as part of normal run details. This information gives you visibility into where time in Robot Framework runs is actually spent. It tracks times on 3 levels: suites, tests and keywords. This means that you can easily visualize what tests are consuming most of the time, and also which keywords are the most time-intensive.

Let's jump into Robot Framework Browser tests that were introduced in Optimizing the CI/CD Pipeline for Robot Framework Browser Using Pabot and RF Logs and see if we can spot anything to improve.

Analyzing the Shards

Robot Framework Browser tests are split into 4 shards and we can look at each shard separately.

Shard 1 Recent Results (153 tests)

Shard 2 Recent Results (290 tests)

Shard 3 Recent Results (218 tests)

Shard 4 Recent Results (106 tests)

Identified Optimization Targets

From analyzing the timing data, several high-impact optimization opportunities emerged:

1. Static Sleep Operations
2. File Generation Performance
3. Browser Server Launch Strategy

Let's look at how we can improve each of these.

1. File Generation Optimization

The original implementation had performance issues:

Generate Test Text File
    [Arguments]    ${length_of_text}
    ${filename} =    Set Variable    ${length_of_text}.txt
    ${length_of_text} =    Convert To Integer    ${length_of_text}
    ${text} =    Generate Random String    length=${length_of_text}
    OperatingSystem.Create File    ${CURDIR}/${filename}    ${text}
    RETURN    ${filename}

When called with a large input like:

${file_name} =    Generate Test Text File    ${10 000 000}

The Generate Random String keyword was taking almost 10 seconds. The solution was to use Python's built-in functions and proper path handling:

Generate Test Text File
    [Arguments]    ${length_of_text}
    ${filename} =    Set Variable    ${length_of_text}.txt
    ${length_of_text} =    Convert To Integer    ${length_of_text}
    ${full_path} =    Normalize Path    ${CURDIR}${/}${filename}
    Evaluate    open(r'${full_path}', 'w').write(''.join(random.choices(string.ascii_letters + string.digits, k=${length_of_text})))    random,string
    RETURN    ${filename}

Key improvements:

• Uses Python's fast random.choices() instead of Robot Framework's string generation
• Direct file writing without intermediate string storage
• Reduced execution time from ~10s to ~2s

2. Browser Server Launch Optimization

The original implementation used a static sleep:

Launch Browser Server Via CLI
    ${python} =    Get Python Binary Path
    ${process1} =    Start Process
    ...    ${python}
    ...    -m
    ...    Browser.entry
    ...    launch-browser-server
    ...    chromium
    ...    headless\=${HEADLESS}
    ...    port\=8277
    ...    wsPath\=server2
    ${process2} =    Start Process
    ...    ${python}
    ...    -m
    ...    Browser.entry
    ...    launch-browser-server
    ...    chromium
    ...    headless\=${HEADLESS}
    ...    port\=8273
    ...    wsPath\=server3
    Sleep    10s
    Connect To Browser    wsEndpoint=ws://localhost:8277/server2    browser=chromium
    # ... rest of the test

The improved version replaces the static sleep with dynamic waiting:

Launch Browser Server Via CLI
    ${python} =    Get Python Binary Path
    ${process1} =    Start Process
    ...    ${python}
    ...    -m
    ...    Browser.entry
    ...    launch-browser-server
    ...    chromium
    ...    headless\=${HEADLESS}
    ...    port\=8277
    ...    wsPath\=server2
    ${process2} =    Start Process
    ...    ${python}
    ...    -m
    ...    Browser.entry
    ...    launch-browser-server
    ...    chromium
    ...    headless\=${HEADLESS}
    ...    port\=8273
    ...    wsPath\=server3
    Wait Until Keyword Succeeds
    ...    10s
    ...    1s
    ...    Connect To Browser
    ...    wsEndpoint=ws://localhost:8277/server2
    ...    browser=chromium
    # ... rest of the test

Key improvements:

• Replaces static 10s sleep with dynamic waiting
• Only waits as long as necessary for the server to be ready
• Maintains reliability while improving speed

Value of Timing Statistics

The timing statistics feature in rflogs.io provides several key benefits:

1. Clear visibility into where test execution time is spent
2. Identification of both frequent slow operations and one-off bottlenecks
3. Ability to validate optimization impacts
4. Easy sharing of performance data with team members

You can try this right now on your own test runs. Just upload your run information to rflogs.io and start exploring. The timing statistics will help you identify the most impactful optimization opportunities in your test suite.