User Acceptance Testing (UAT) Status

Market Plan is returning high reach % results for certain fleet and roadside packages

Root cause confirmed; update applied to QA on 12/22 and tested OK (pending promotion to staging environment)

READY FOR RETEST (passes Geopath testing)

API 2.2 performance is slower than expected

Attempting additional performance improvements

READY FOR RETEST (passes Geopath testing)

Market Plan is returning low reach % results for certain fleet packages

Root cause confirmed (configuration changes and API implementation issues); additional data-level issues are addressing LinkNYC packages; supplemental processing ran 12/20 & 12/21 and tested OK in QA & STAGING on 1/4/23

READY FOR RETEST (passes Geopath testing) [Note: Test cases using market plan cannot exceed package size of 1500 spots]

Certain spots are showing reach that is too high (at unit level)

Data patch is currently being processed during the evening of 12/19; supplemental processing ran 12/20 & 12/21 and tested OK in QA & STAGING on 1/4/23

READY FOR RETEST (passes Geopath testing)

Certain packages are showing reach that is too high (when certain spots with reach that’s too high are included)

Data patch is currently being processed during the evening of 12/19; supplemental processing ran 12/20 & 12/21 and is now undergoing testing in the QA environment before promotion (additional variations are being addressed)

READY FOR RETEST (passes Geopath testing)

Certain place-based inventory is showing low reach & impressions counts

Data patch is currently being processed during the evening of 12/19; supplemental processing ran 12/20 & 12/21 and tested OK in QA & STAGING on 1/4/23

READY FOR RETEST in all environments (passes Geopath testing)

Multiple P0/P5/P18/P21+ audiences are shown in the Geopath Insights Suite (in staging)

Geopath UI Team implemented multiple types of P0+ audiences for testing/validation purposes and did not remove them from the staging build

READY FOR RETEST (passes Geopath testing); CONFIRMED RESOLVED by UAT group

Target audiences beyond P0/P5/P18/P21+ are shown in the Geopath Insights Suite (in staging) but are not functional

UI Team implemented all expected audiences shown in https://geopath.atlassian.net/wiki/spaces/GAP/pages/2220818433 prior to all supporting data available

READY FOR RETEST (passes Geopath testing)

Direct API request to /inventory/search endpoint for organization-wide queries returns a ‘417’ error

API environment configuration-related issue (developers are debugging)

READY FOR RETEST (passes Geopath testing); CONFIRMED RESOLVED by UAT group

API /inventory/search/summary (and potentially inventory plans with reach goals) that use multiple summarization criteria will show aggregate (summed) results rather than the units that meet all summarization criteria

API bugfix applied in QA & STAGING environment and tested OK

READY FOR RETEST (passes Geopath testing)

‘representations’ array in /inventory/measures/summary is returning NULL values

API bugfix applied in QA & STAGING environment and tested OK

READY FOR RETEST (passes Geopath testing)

/inventory/search/summary against ‘all US population' scenarios shows an in-market population 44% greater than the USA (476M vs 333M)

API investigation complete + fix applied/validated in STAGING

READY FOR RETEST (passes Geopath testing)

Total population in market plan UI appears to be larger than the DMA supports

UI vs API investigation complete; data fix applied

READY FOR RETEST (passes Geopath testing)

Fleet reach/impression metrics in NYC show variances between 5 boroughs (as counties) vs NYC DMA [elevated to high severity on 2/8 as issue may impact all media types when using multi-geography plans]

UI vs API investigation underway (examining market plan vs inventory plan; Fix applied to QA environment, supplemental update applied on 2/8 and retesting of both fixes now underway

READY FOR RETEST in STAGING (passes Geopath testing)

Package containing a single spot will render different result in inventory plan vs a single spot in search/summary [low severity; fleet only]

Search/summary API design calls for returning a maximum amount of ‘frames’ (ad units across ALL spots) vs a single frame as the default output; Fix applied to QA environment and retest complete

READY FOR RETEST (passes Geopath testing)

‘Total population’ column is mislabeled in market plan UI (rather than ‘target market population’) [low severity]

UI fix underway (and expedited), will investigate as a candidate for release next week (given low priority)

READY FOR RETEST (passes Geopath testing)

Operator-specific spot-specific, construction-specific, face-specific and frame-specific IDs are not consistently available in staging data

Operator-specific IDs were partially loaded into API v2.2 inventory data schema (e.g., plant_spot_id becomes spot_id_alias_primary, etc)

READY FOR RETEST (passes Geopath testing)

/inventory/summary/measures returns an overall reach_net value that matches the largest batch of spot_ids (rather than exceeds the value of the largest batch of spot_ids as expected)

API investigation underway to determine if data-level or API-level issue persists; API patch is applied to QA and is being tested

READY FOR RETEST (passes Geopath testing); clarifying point: in scenarios where one ‘batch’ of spot_ids has a greater reach than another ‘batch’ and audience duplication likelihood is large, the largest reach net & % is returned

Approximately 10K spots (out of nearly 1.5M) show net_reach = 0

Data-level fix underway to address conditions where unit-level reach is null or 0 for inventory in ‘published-measured’ status

READY FOR RETEST (passes Geopath testing); now, only 30 spots with <7 people/week reach show a reach_net value rounded down to 0

Named searches fail to generate if more than 500 spots match the search criteria

API-level & environment-level investigation underway; database issue identified and addressed, API-level caching issue identified and addressed, caching & authentication issues under investigation

READY FOR RETEST (passes Geopath testing)

inventory/search results may return 500 errors when page_size values >=1000 are used and/or if 1500+ frame_ids/spot_ids are used as input parameters

API-level & environment-level investigation complete; database issue & configuration issue identified and remediation applied across environments

READY FOR RETEST (passes Geopath testing)

market plans endpoint (/inventory/plans) no longer accepts ‘calc_period_days’ as a parameter

Issue confirmed in staging environment; fix applied to QA environment

READY FOR RETEST in all environments (passes Geopath testing)

market plan queries submitted with specific inventory parameters will yield different results each time the market plan is run

Issue confirmed in staging & QA environments; confirmed root cause and onshore data team working to resolve

READY FOR RETEST in all environments (passes Geopath testing)

reach and frequency may fluctuate at different package size levels for roadside media

Roadside reach model is an estimate, which is undergoing continuous testing for improved consistency between package sizes. In addition, USA-wide reach estimates require supplemental data updates for accuracy.

READY FOR RETEST in all environments (passes Geopath testing)

certain target audiences run against single spots in inventory plan (via Insights Suite UI) will yield 0 & N/A results (workaround is to use Explore module and filter on single spot for target audience)

Code fixes have been tested OK in lower environments and has been promoted to STAGING & PRODUCTION

READY FOR RETEST in all environments (passes Geopath testing)

Hispanic audiences 18+ / 21+ are only available as more granular audience segments, no aggregate audience (e.g., ‘Hispanic persons 18+’) exists for 2023 data (as context: 200+ additional target audiences will be available by EOM March)

Works as designed; not a defect, but an requirement not previously addressed in the target audience list for 2023. Incremental 200 audiences have been promoted to STAGING & PRODUCTION APIs, with UI updates to occur on 3/27

READY FOR RETEST in all environments (passes Geopath testing in API; Production UI updates to include new audiences scheduled for 3/27)

market plans show lower reach % in 5 boroughs of NYC compared to DMA (the reach % in 5 counties of NYC should be higher than the DMA)

Code fix has been tested OK in lower environments and has been promoted to STAGING & PRODUCTION

READY FOR RETEST in all environments (passes Geopath testing)

certain large market plans reach market-level maximums (e.g., 95% of DMA)

Code fix has been tested OK in lower environments and has been promoted to STAGING & PRODUCTION

READY FOR RETEST in all environments (passes Geopath testing)

when unpublished spots or spots with no reach are included in a batch requested from inventory/measures/summary, a 500 error is returned

Code fix has been tested OK in lower environments and has been promoted to STAGING & PRODUCTION

READY FOR RETEST in all environments (passes Geopath testing)

when using the UI, some market plans that include ‘operator’ parameter won’t show results despite rendering a nonzero count of ‘total spots’ for the selected criteria

Root cause of issue identified (UI is passing incorrect parameters to API; have deployed a hotfix into production & staging UIs)

READY FOR RETEST in all environments (passes Geopath testing)

/inventory/measures/summary times-out when too many batches of spots are submitted for aggregation

Infrastructure and API performance enhancements have been applied to extend performance on /inventory/measures/summary endpoint (should increase the maximum number of spots per batch supported as well as the number of batches allowable per API call)

READY FOR RETEST in all environments (passes Geopath testing) [technical note: a workaround exists, as API users can submit multiple batches via multiple API calls in the event of timeouts]

inventory statuses of some units are incorrectly shown in the insights suite (e.g., “audit level 1”)

relational data update with locations and statuses was not universally applied in all circumstances and user interface ‘map tiles’ were not completely updated

READY FOR RETEST in all environments (passes Geopath testing)

for place-based and fleet market plans with a large count of units, reach %s approach market-level maximums (e.g., 85% of DMA)

place based & transit reach boundaries are undergoing continuous testing for improved consistency between package sizes; updates applied in QA & STAGING

READY FOR RETEST in all environments (passes Geopath testing)

retrieving place-specific audience measures via inventory/search/summary by summarizing by place (rather than using place_name as an input) yields reach and population values that represent all the DMAs returned by the search query rather than the DMA containing the specific place

Code has been investigated and enhancement request has been applied in both QA and STAGING environments

READY FOR RETEST in all environments (passes Geopath testing)

when computing roadside R/F for some types of roadside panels, reported reach % may be excessively high or excessively low

Root cause of issue identified and resolved (API code-level issue)

READY FOR RETEST in all environments (passes Geopath testing)

certain large market plans in scheduled fleet scenarios report excessively high reach

Root cause of issue identified and resolved (API code-level issue)

READY FOR RETEST in all environments (passes Geopath testing)

start and end dates for a 1-day time period requested via /inventory/measures/summary, when spanning multiple unique calendar dates, will return audience measures that represent multi-day periods

API code & related logic has been updated and a fix has been applied in Geopath’s QA environment (will be promoted to staging during the first week of May)

READY FOR RETEST in all environments (passes Geopath testing)

local_dma produces different R/F outputs than using equivalent DMA on inventory/search/summary

Root cause of issue identified and resolved (API code-level issue)

READY FOR RETEST in all environments (passes Geopath testing)

inventory plans, market plans, and inventory/search/summary for a single inventory unit should return the same measures as inventory/spot/{spot_id}

Root cause of issue identified and resolved (API code-level issue)

READY FOR RETEST in all environments (passes Geopath testing)

inventory/measures/summary returns 500 error when running two spots from the same frame

Root cause of issue identified and resolved (API code-level issue)

READY FOR RETEST in all environments (passes Geopath testing)

market plans for some scheduled fleet networks will not return audience measures

Code-level review has completed and a fix has been applied in all environments

READY FOR RETEST in all environments (passes Geopath testing)

market plans for some scheduled fleet networks will return negative numbers for audience measures

Code-level review has completed and a fix has been applied in all environments

READY FOR RETEST in all environments (passes Geopath testing)

market plans for certain roadside unit types (digital urban panels, digital video panels, urban panels) will not return audience measures

Code-level review has completed and a fix has been applied in all environments

READY FOR RETEST in all environments (passes Geopath testing)

market plan using 3+ separate media class aggregation breakouts may time-out

Code-level review has completed and a fix has been applied in all environments

READY FOR RETEST in all environments (passes Geopath testing)

inventory plans for some CBSAs will not return audience measures

Code-level review has completed and a fix has been applied in all environments

READY FOR RETEST in all environments (passes Geopath testing)

some R&F metrics generated by the inventory/summary/measures (inventory plan) do not match inventory/plans (market plan), inventory/search/summary (explore), and/or inventory/spot/{spot_id} (spot-level details)

API-level & data-level investigations completed and fix applied; remaining mismatches occur in <2.5% of cases, likely due to API-level logic; investigation into edge cases continues

PARTIALLY RESOLVED IN ALL ENVIRONMENTS (GEOPATH CONTINUES INVESTIGATING THE REMAINING SCENARIOS WHERE MISMATCHES OCCUR; THIS SMALL MISMATCHING % SHOULDN’T BLOCK INTEGRATIONS)

retrieving aggregate audience measures where a specific market (or collection of markets) is unspecified, and the implication is 'aggregate all inventory across the USA’, reach numbers may appear too low

Code has been investigated and enhancement request is in progress

NO ACTION REQUIRED YET (fix is in progress and awaits deployment/testing in STAGING environment)

named searches (i.e. downloadable results in JSON format) for very large spot counts (e.g., all roadside spots) do not always consistently deliver downloadable outputs

Infrastructure and API code-level optimizations are under investigation; multiple environment configuration updates applied to be more performant

NO ACTION REQUIRED YET (reasonable scaling has been applied to production environment but users cannot export ALL inventory in one request; guidance is to break-down named searches by DMA for best results)