Challenge Report

You need to make changes but want see how they would impact live applications. You implement new rules, but want to verify that they are working as intended. The Challenge Report allows you to test out changes by promoting them to the Test environment, re-decisioning live decisions, and comparing the results side-by-side (see a Sample Challenge Report).

Prerequisites

In order to run a Decision Challenge Report, applications must have been processed by Decision Manager long enough to have a pool of applications to review changes against. (For example, you would not be able to run an effective Decision Challenge Report against only a few dozen applications.)

To Submit a Challenge Request

1. Make changes to a product and promote the changes to the Test environment.

2. From the Reporting and Analysis area click on the Decision Challenger button.

3. Complete the Decision Challenger Request as follows:

•	Give the challenge request a meaningful name such as "Tier 2 Cutoff Change 07/31/17".

•	Select the "Test Product" just promoted to Test.

•	Enter a Date Range or a Max Number of decisions to be considered. (See IMPORTANT NOTES #1)

•	The max decision count that can be considered is 20,000. If not provided on input, or if provided input is greater than 20,000, then the max count is set to 20,000. The report will not pull more than 20,000 records.

•	If desired, you can further limit your results by selecting only applications originally underwritten by a specific Live Product or multiple Live Products.

•	Submit the request (See IMPORTANT NOTES #3)

To View a Challenge Report Show Me (4:23)

1. After a reasonable amount of time (usually 30 minutes or less), select the CUDL Report Viewer from Reporting and Analysis.

2. From the menu bar at the top of the report viewer, select Decision Manager > Challenger Summary.

3. Select the Request Name that you assigned the challenge from the pick list and click Submit Report.

4. Review the Challenger Summary Report to see how the changes impacted the Decision, Loan Amount, Tier.

5. To view the Challenge Detail Report, click on Quick Reports, then click on Challenge Detail Report, then select the specific report from the Challenge Request Name pick list.

IMPORTANT NOTES:

1.	Records for the report are pulled as follows:

•	If a maximum count has not been provided OR if it’s greater than 20,000, then the maximum is set to 20,000. So, the report will never allow more than 20,000 records (1 record = 1 decision)

•	The pool of records is ordered by the decision date. It will get the most recently decisioned applications up to the maximum count.

•	Then, from that initial pool (based on maximum count), the Date Range filter is applied, and the Live Products filter is applied. Not all applications in the date range will necessarily be considered in the report. Records in the date range can be excluded because they are beyond the maximum count.

•	Additionally, to be included in the report, the record must pass the routing criteria of the Test Product.

2.	Submitting a request queues up the applications selected for re-decisioning. You will need to periodically check the CUDL Report Viewer to see if the challenge request you submitted has completed its run.

New releases of Decision Manager may include new variables that were formerly unavailable. Although these variables are immediately available for use in the Live environment, applications processed prior to the release will not have these variables. Therefore, Challenge Requests that include new variables will not be possible on applications processed prior to the release of those variables.

4.	The Challenge Report may include multiple rows with the same application number. This occurs when an application is re-decisioned. Each decision is a record in the Challenge and will be re-decisioned and included in the Challenge Report.

Page url: https://decision.cudl.com/help/index.html?challenge_reports.htm