FinTOC-2019 Shared Task:

“Financial Document Structure Extraction”

NEW: Shared task data format and evaluation metrics description Data Format Details

NEW: Shared task registration is now open (FinToc Team Registration Form)

NEW: Shared task discussion group (FinToc Google Group [Active]). Use this to discuss topics re the shared task.

 

Updates:

  • We will continue collecting runs for both shared tasks until the 2 August. 
  • The results will be published 5 August together with the testing set.
  • Deadline for papers 10 August (anytime on earth) and reviews and notification of acceptance will be the 17 August. 

 

Paper submission

The format for the paper is exactly the same as for the general workshop, although we favor papers of 4 pages, you can find all the details in the workshop web page

Regarding the submission webpage it is also the same as the workshop: https://easychair.org/conferences/?conf=fnp2019

 

*******The only difference is the title of the paper**********

Please use the following naming:

[Name of the team]@FinTOC-2019 Shared Task

 

Registration:

Please register your team using the google form below:

FinToc Team Registration Form

 

Introduction:

A vast amount of financial documents are created and published constantly in machine-readable formats (generally PDF file format), with only minimal structure information. Firms use such documents to report their activities, financial situation or potential investment plans to shareholders, investors and the financial markets, basically corporate annual reports containing detailed financial and operational information.

In some countries as in the US or in France, regulators as EDGAR SEC or AMF require firms to follow a certain template when reporting their financial results to insure standardisation and consistency across firms’ disclosures. In other European countries, on the other hand, the management usually have more discretion on what where and how to report resulting in lack of standardisation between financial documents published within the same market.

In this shared task, we focus on analysing Financial Prospectuses; official PDF documents in which investment funds precisely describe their characteristics and investment modalities. Although the content they must include is often regulated, their format is not standardized and displays a great deal of variability ranging from plain text format, towards more graphical and tabular presentation of data and information. The majority of prospectuses are published without a table of content (TOC), which is usually needed to help readers to navigate within the document by following a simple outline of headers and page numbers, and assist legal teams in checking if all the contents required are fully included. Thus, automatic analyses of prospectuses to extract their structure is becoming more and more vital to many firms across the world.

Task:

As part of the Financial Narrative Processing Workshop, we present a shared task on Financial Document Structure Extraction.

Systems participating in this shared task will be given a sample collection of financial prospectuses with different level of structure and different lengths (document sizes), which are to be automatically analyzed to extract structural information and build a table of content.

The task will contain two sub tasks are:

a) Title detection

This is a binary classification task aiming at detecting titles in financial prospectuses. Given a set of text blocks, the goal is to classify each given text block as a ‘title’ or ‘non-title’. As shown in Figure 1 the titles can have different layouts (marked with red and green boxes) and they have to be distinguished from the regular text (‘non-title’ with grey boxes).

Click to show full sized image.

b) TOC structure extraction

The TOC is a hierarchical organisation of the headers of a document. In this subtask, we provide only the headers of a prospectus, and the goal is to (i) identify the hierarchical level of the header (ii) organize the headers of the document according to this hierarchical structure. Note that two headers, with the same layout and the same text can have different hierarchical levels depending on their location in the document.

Participants need to register. Once registered, all participating teams will be provided with a common training dataset, which includes common pre-processed input and corrected output. A common development set will also be provided. A blind test data set will be used to evaluate the output of the participating teams. An evaluation script will be provided to all the teams. In addition to the PDF version of the documents, we will provide their XML representation.

Background:

Existing work on book and document table of contents (TOC) recognition has been almost all on small size, application-dependent, and domain-specific datasets. However, TOC of documents from different domains differ significantly in their visual layout and style, making TOC recognition a challenging problem for a large scale collection of heterogeneous documents and books. Compared to regular books (mostly provided in a full text format with limited structural information such as pages and paragraphs), Financial documents, containing textual and non textual content, have a more sophisticated structure including, parts, sections, sub-sections, sub-sub-sections. 

Data Format and Evaluation:

The following pdf file describes the data format and evaluation metric used in the shared task: Data Format Details

 

Important Dates:

(suggested plan FNP FinTOC task at NoDaLiDa 2019)

  • March 25, 2019: First announcement of shared task
  • April 10, 2019: set up of shared task website
  • April 15, 2019: registration begins and release of initial training sets and scoring script
  • May 18, 2019: Final training data release
  • Jun 29, 2019: registration deadline
  • July 6, 2019: test set available
  • July 13, 2019: systems’ outputs collected (extended to 2 August)
  • July 20, 2019: system results due to participants (extended to 5 August)
  • July 27, 2019: shared task system papers due (extended to 10 August anytime on earth)
  • Aug 17, 2019: notification of acceptance
  • Aug 24, 2019: camera ready version of shared task system papers due
  • Sep 30, 2019: Workshop day

Shared Task Organisers:

Shared Task Contact:

Questions about FinTOC-2019 shared task can be sent to:

fin.toc.task@gmail.com