General process issues
Q: I have uploaded my PSD file and the data item is still showing 'no data'?
A: You need to update this manually as this lets us know you have completed your PSD reporting for the quarter:
- In Gabriel, click on the ‘view schedules’ screen
- Click on the ‘return due 26/10/2012’ schedule
- Put a tick in the ‘select item’ box next to PSD001/5 and then click on the ‘submit’ button
Q: I have tried to validate the PSD001/5 data item but it is not working?
A: You do not need to validate the PSD001/5 data item. Just tell us you have completed your PSD reporting by putting a tick in the ‘select item’ box next to PSD001/5, and then click on the ‘submit’ button.
Q: I do not have any transactions to report, how do I submit a nil return?
A: By updating the PDS001/5 data item as ‘submitted’:
- In Gabriel, click on the ‘view schedules’ screen
- Click on the ‘return due 26/10/2012’ schedule
- Put a tick in the ‘select item’ box next to PSD001/5 and then click on the ‘submit’ button
Q: When I click the 'submit' button it asks me to confirm that there are no reported transactions for the period, despite the fact that I have uploaded a file with transactions in it and I can see it in the ‘report history’ area?
A: It’s just a timing issue - the system processes the transactions received via XML files overnight and updates the report as ‘submitted’. If you updated the PSD001/5 data item the next day the count would be updated.
Q: I have submitted my XML file but some transactions have been rejected because 'Customer date of birth field must match comparison: entry must indicate age between 18 and 115 inclusive otherwise must be omitted'?
A: The system has rejected these due to the business validation rule that is currently being applied to the date of birth being reported. In this instance, to load the transaction please remove the date of birth information and the tags themselves from the XML and resubmit.
Q: I have tried to upload my XML and it has displayed an error/not worked. I was advised that the PSD XML had not changed?
A: You are using the wrong screen in Gabriel. You need to click on ‘product sales data’ in the left hand side and use the PSD XML upload screen here instead.
Q7: I uploaded my file but its still not showing as submitted
A: Any files loaded into Gabriel initially create a record on the report history screen as ‘received’. At this point any XML schema issues will be identified. Then Gabriel processes the transactions on the files overnight and posts the results back into the report history screen. The report will then have been updated to ‘submitted’ and a response file should have been sent back to the firm.
Q: I have submitted my XML file via SFTP but still cannot see this in the report history in Gabriel?
A: It is probably because the file has not yet been picked up by Gabriel. In terms of the SFTP process timings, they are as follows:
- Gabriel checks at 9am, 1pm and 5pm for any new files sent by SFTP to the TRS server (and passes any response files back during these windows)
- Any files found are loaded into Gabriel, and a corresponding record is created on the report history screen as ‘received’. At this point XML schema issues will be identified
- Gabriel processes the transactions on the files overnight and posts the results back into the 'report history' screen. The report will then have been updated to ‘submitted’ and a response file should have been sent back to the firm
Q: I have amended the rejected transactions, but the report history is not removing them from the report the transaction was originally reported on?
A: These counts are not updated as we would lose the original submission issues and would not see when and why a transaction was initially rejected.
Q: I have submitted my XML file via SFTP but still cannot see a response file?
A: It is probably because the file has not yet been picked up by Gabriel. In terms of the SFTP process timings, they are as follows:
- Gabriel checks at 9am, 1pm and 5pm for any new files sent by SFTP to the TRS server (and passes any response files back during these windows)
- Any files found are loaded into Gabriel, and a corresponding record is created on the report history screen as ‘received’. At this point XML schema issues will be identified
- Gabriel processes the transactions on the files overnight and posts the results back into the 'report history' screen. The report will then have been updated to ‘submitted’ and a response file should have been sent back to the firm
Q: I have submitted my XML file but some transactions have been rejected because 'transaction reference - the transaction has been rejected as a cancellation has been found for the same transaction within the report'?
A: The system has identified that the transaction has been reported and then cancelled in the same report. In this scenario you do not need you to report these to us.
Q: I have submitted my XML file but some transactions have been rejected because 'original transaction not found for cancellation'?
A: The system has not been able to find the transaction to cancel from the data provided in the XML report. Please use the ‘search transactions’ screen in Gabriel to find and cancel the transaction manually if applicable. If you cannot find the transaction then no further action is required, ie it’s effectively already cancelled.
Input/transfer method
Q: How do I decide which transfer method to use?
A: There are two basic methods to input transaction data: keying the data in via web forms or creating an offline XML file.
The off-line XML method has two different processes: web upload (https) or system to system (SFTP).
The XML files used in both of these approaches are identical in structure, only the method of transfer differs.
Keying in data via web forms (web input)
Under this method, you enter transactional data into a small number of TRS PSD forms accessed through a web browser. As you enter each transaction the fields are validated, and any errors are displayed immediately.
Because you need to input the data manually, this method is betted suited to firms:
- with lower volumes of data and/or without any integrated back-office system
- that typically use a dial-up method of communication
- that probably do not have a detailed IT understanding or a dedicated IT resource/computer support person available in their organisation
Offline XML input (web upload or system to system)
There are several ways an XML file containing transactional data may be created. Firms may have:
- an existing back-office system that has been modified to produce the XML file
- procured a new system to produce the XML file
- identified a method of manually creating an XML file from current system data, by exporting from Excel for example
Having created an XML file, you have to decide whether to use web upload or system to system to transmit this XML file to Gabriel. You do not need to do this purely because of volumes, you may decide to use one approach because it is more convenient.
The advantage of the system to system approach is that the data transfer can be initiated by a form of automated script that can be scheduled to run at the appropriate time. For example, it could be that an XML file of transactions is assumed to have been captured in a known location at a specific point in time ready for the automation script to be enabled to transfer this file to us. This method would not be size constrained, as it could be used to transmit any size file.
Web upload can be used to manually send any XML file. However, there is a file size threshold, at which point the web upload becomes too slow. It is not possible at this time to put an exact figure on this, and it depends on the product type you are reporting on and the type of communications approach you are using. For guidance, in data-sizing terms pure protection is very small and mortgages is significantly larger. In file transmission terms, a dial-up modem is very slow, broadband or fibre optic connections can be very fast.
System to system approaches tend to be more efficient as they compress the data to make the size of the data file to be transmitted as small as possible, and so decrease the file transfer time.
If possible, we may publish further guidance on indicative transmission time estimates for each method and their impacts on equipment. However, firms need to decide for themselves the impact of introducing TRS PSD into their organisation.
We positively encourage you to verify the validity of your XML file by running the file through an XML parser. This will avoid unnecessary delays in uploading XML files that may contain errors that need to be resolved, and ultimately reduce the amount of future effort in resolving data errors.
For PSD only: to avoid long transmission times, PSD files could be uploaded on a periodic basis rather than waiting until the 20-day reporting period to submit the data from a complete quarter.
Web upload
For this, you generate an XML file outside TRS PSD and then use TRS PSD to upload the file. Through the web browser, you can access the TRS PSD XML upload screen and the browse facility used to navigate to the appropriate path and file to identify the file to be uploaded. Once you have submitted the file, TRS PSD will validate it and, when the validation process has finished, will provide feedback while you are still online. This approach is often described as ‘synchronous’.
For users of Internet Explorer, within the browser window, the status bar will show the progress of the upload, and in all browsers a series of dots displayed will show that the validation process is active.
For the reasons already explained, this type of input could be considered by firms:
- with only moderate volumes of data, with some form of back-office system or package that generates an XML file
- with some form of medium-speed communication, such as broadband
- that probably have a basic IT awareness or limited access to a dedicated IT resource available in their organisation
System to system
This also involves generating an XML file outside the TRS PSD application, but it uses an external method of file upload and download of the validation results using secure file transfer protocol (SFTP). Each of these steps are distinct phases – the upload of data and the download of results are separate SFTP sessions. Between these sessions, firms must allow sufficient time for the file to be validated and parsed. This approach is very often described as 'asynchronous'. See the other questions covering the detail of the SFTP process and downloading validation results.
This type of input could be considered by firms:
- with large volumes of data, generally with some form of back-office system
- with either a medium/high-speed communication such as broadband or fibre optic connection
- that probably have a dedicated IT resource available in their organisation
The major benefit of this approach is that the data file transmitted as part of the SFTP process is compressed. So the transmission time is significantly shorter.
To avoid doubt, we have used the terms ‘lower’, ‘moderate’, and ‘large’ as an indicative sizing threshold, and firms should decide for themselves the real threshold based on their own profiles.
Q: I've identified my preferred method of transferring data to Gabriel – what do I do now?
A: If you are planning to use an XML method of data input, you should download both the XML schemas and a copy of an XML parser/validation process, so your files can be validated offline before uploading to us. For system to system, you will also need to acquire a copy of a secure file transfer protocol (SFTP) client. Your method of data input will be agreed as part of registration.
Q: How do I download the FCA Gabriel PSD schemas?
A: XML schemas can be found in the relevant PSD001 to PSD005 data reference guides.
Q: How do I download a copy of an XML parser/validation tool?
A: Every XML report file that is sent to the Gabriel must be validated against the relevant TRS PSD XML schema available on our website, so it is necessary for firms to acquire appropriate software. We cannot recommend a particular software package, but the following link lists some of the more well-known versions: www.garshol.priv.no/download/xmltools/cat_ix.html#SC_XMLVal[1]
Q: What do I need to use the system to system interfaces?
A: You will need a secure file transfer protocol (SFTP) client, software that is generally available in two forms. One version uses a Windows interface, whereby the screen is typically (although not always) divided into two windows, and you transfer files to us by dragging files with a mouse. The other version uses some form of command-level interface, by which a call to STFP may be performed as a script. We cannot recommend a particular package, but the following links describe some of the free versions of this software: Free FTP clients, secure FTP (SFTP) programs[2] or OpenSSH[3].
Q: Are there any specific configuration settings I need to know about to communicate with the SFTP server?
A: If you have expressed an interest in using this method of communication, we will contact you during the rollout period to let you know the settings for connecting to the trial area. Once you have certified, you will be given the setting for the main application.
Q: Are there any restrictions on the file size submitted via web upload?
A: TRS imposes a limit of approximately 35MB for files submitted within a web upload XML report. Other factors – such as speed of connection, session timeout and current load on the system – will restrict the maximum size from a practical point of view, and because of this submission should be aimed to be completed within 30 minutes.
Q: Is the method of uploading multiple files from a single location acceptable?
A: Multiple files are acceptable from a single system, as long as the return indicates unique identifiers for each file.
Q: Does the use of XML affect data transmission speeds?
A: Using XML can increase the size of the file being transmitted – by 8 to 15 times the actual data content. This will affect the time the transmission takes, along with other environmental factors such as the size and type of communications link used. Firms need to decide for themselves the impact of introducing TRS PSD into their organisation.
Q: What does an SFTP upload script look like?
A: The following is an example of an SFTP upload script:
open trs.fsa.gov.uk
15782
Abc123
cd report_upload
lcd trs_transmit
put filename.xml
quit
For more information, see the system to system interface guide[4].
Q: What does an SFTP script to download validation results look like?
A: The following is an example of an SFTP validation script:
open ftp.trs.fsa.gov.uk
15782
Abc123
cd result_download
lcd trs_results
get filename_rpt.xml
quit
For more information, see the system to system interface guide[4].
Q: How secure is my data during the file transmission process?
A: All data transmitted, regardless of approach, will use a secure method of transfer. Web input and web upload will use https and system to system will use SFTP (as opposed to http and FTP, the ‘S’ indicating a secure connection). These secure methods of transfer use encryption processes before sending the data and appropriate decryption processes at the receiving end. So the communication channel performs the encryption and not any proprietary vendor software outside of TRS PSD.
Further technical reference and protocol information is available on:
www.w3.org/Protocols/rfc959/[6]
Q: (PSD only) We intend to send information for retail and mortgage products – do you require two separate XML documents?
A: Each product type will require a separate XML file, so if you have to report RI and MG you will need to provide Gabriel with two returns and therefore two XML files.
Transfer acceptance/validation/error handling
Q: How do I know whether an XML file uploaded via the system to system interface has been accepted by Gabriel, or whether it contains any errors?
A: The system to system method of data transfer involves a physical disconnect between the two stages (ie XML upload and the validation report download). So following transfer and disconnection, there is a requirement to reconnect to download data validation and data transfer results.
This information, in the form of a validation results XML file, will be produced detailing the results of the XML upload and any errors identified. This file will be available for download and local processing once the validation process has completed overnight. Please refer to the validation replies schema and the validation replies data reference guide for more information, which can be found in the relevant PSD001 to PSD005 data reference guides section.
Alternatively you can log on to Gabriel and view the submitted reports (and any validation errors/transaction rejections) in the Gabriel PSD report history screen.
Q: At what point can I check for transmission results?
A: Web upload schema results will be reported back automatically once the upload and validation have completed and been processed overnight. System to system upload will require you to query the data transfer results. There is no alert mechanism to inform the sender that a file has completed verification. So when using the system to system interface, you should wait a day after uploading before requesting a subsequent download of the validation results.
Q: How quickly is the Gabriel validation report generated back to us? Is it immediate?
A: The answer depends on the method that you use. Some validation will take place at the time of data entry, eg if you enter data via web forms. The turnaround once submitted will depend on the size of the data file and the number of errors encountered. The design aim is to respond within 30 minutes, but this has still to be tested and confirmed.
Q: Do we resubmit just the corrected/cancelled records or the entire file (within the same period)?
A: Resubmit only corrected records.
Q: Do errors have to be corrected and resubmitted in the same period, or can they be included in the submission for the next period?
A: Submissions will need to be corrected and resubmitted in the same period.
Q: What happens if we send an accepted file more than once in error?
A: Sending an accepted file more than once will cause duplicates to be created, and the file will be rejected.
Q: Will the FCA report failures on all fields which fail in the record, or just the first one it finds?
A: It depends on the nature of the failure. Assuming that the system is able to read the file and is able to continue to the end, all errors encountered will be reported.