MD2466 London Development Database Automation & Live Data Hub project

Type of decision: 
Mayoral decision
Date signed: 
29 July 2019
Decision by: 
Sadiq Khan, Mayor of London

Executive summary

This Mayoral Decision (MD) seeks approval to fund the development of an up-to-date, automated data collection and monitoring system for the London Plan. The data collection and system will: provide a robust evidence base on which to prepare new planning policies and make decisions on planning applications; reduce the cost of collecting development data to local planning authorities and Londoners; and provide a clearer, more transparent picture of what is happening on the ground for Londoners, SMEs, government, and others interested in London’s growth. (Note that this MD funds the second phase of this project; MD2390 funded Phase 1 in the previous financial year.)


That the Mayor approves:

Expenditure of up to £373,000 in 2019/20 on Phase 2 of the project to create an updated, automated London Development Database. In combination with Phase 1 covered by MD2390, the full expenditure for this project is £523,000.

The costs for Phase 2 are as follows:

£99,000 – staff; and
£274,000 – systems.

Part 1: Non-confidential facts and advice

Introduction and background

The Greater London Authority Act 1999 places responsibility for strategic planning in London on the Mayor and requires him to produce a strategic spatial strategy for London, and to keep it under review.

Since 2004 the effectiveness of that spatial strategy has been monitored through the London Development Database and reported through an annual monitoring report. The process requires each of the planning authorities in London to review each permission they grant and report permissions for developments which:

• Create or lose 1 or more dwellings;
• Result in a change of use of more than 1000m2; or
• Result in the loss or gain of public open space.

This data is collected and entered into the current London Development Database (LDD) through a web portal, often manually, and checked by GLA officers.

The data is collected at the following stages:

• When permission is granted;
• When the development commences; and
• When the development is completed.

Data is also collected on when a planning permission expires unimplemented.

Whilst still the only database of its kind - and ground breaking for its time - technology, the planning process, and the development industry have since moved on, not least in the expectations that data will be collected to enable the effectiveness of policies and development on the ground to be assessed much more comprehensively than the limited dataset above.

The Mayor’s draft London Plan is soon to be adopted and the limitations of the current LDD mean that the effectiveness of this critical policy document on non-referable schemes will not be measurable for over a year, if not 18 months, after the policy document has been agreed and published. This is because the manual nature of data collection creates a delay between what is happening on the ground or in the planning system, and when that activity is picked up by monitoring officers and reported to the GLA.

The GLA also has other projects that rely on the LDD for their effectiveness, in particular the London Infrastructure Mapping Application, the soon to be launched accessible housing register, and the work of the City Intelligence team on demographics, schools, and other social infrastructure. The LDD’s current limitations constrain the effectiveness of these tools and analysis.

Specialist services are required therefore, to enable the LDD to be updated to meet the GLA’s London Plan and other needs efficiently.

The update of the London Development Database will address these issues by automating the collection of monitoring data for all planning application types.

This request for a Mayoral Decision aligns with the Smarter London Together roadmap and the GLA’s wider work stream regarding Digital Planning. The project will be delivered to meet the criteria of the Local Digital Declaration of which the GLA is a signatory.

The development of a new system has been informed by a discovery exercise where GLA staff visited 26 Local Planning Authorities (LPAs) in early 2018 to identify: how they currently complete LDD requirements; their existing back office systems and management practices; the cost of the status quo; the quality of the data they provide; and issues they face completing the requirements. A summary of this research can be found at

Objectives and expected outcomes

The objectives of the automation of the London Development Database are to:

• Have a clear, up to date and robust evidence base to prepare new planning policies and make decisions on planning applications;
• Reduce the cost of collecting development data for Londoners and local planning authorities; and
• Provide a clearer, more transparent picture of what is happening on the ground for Londoners, SMEs, Government, and others interested in London’s growth.

The outputs of this project are:

• A central register of all planning applications submitted within the GLA area with key monitoring data in machine-readable format; and
• A dataset that will inform decisions, with published tools to enable Londoners to analyse the data.

There are likely to be other benefits from the delivery of this automation, including:

• E-alert notifications for Londoners of all developments that might affect them, regardless of borough boundaries, enabling boroughs to move away from written neighbour notification letters;
• Opportunities for businesses to innovate based on the live data feed of information, which we hope will stimulate new markets and opportunities for business;
• Enabling smoother delivery of housing by collecting data that allows the GLA to accurately monitor how much is being built, and to assess any blockers in the planning system that may be slowing it down; and
• Encouraging boroughs to use the connected levels of information to undertake more specific monitoring of developments and generate faster payment of the Community Infrastructure Levy and policing of building regulations.

Equality comments

The public-sector equality duty (PSED) under section 149 of the Equality Act 2010 requires the identification and evaluation of the likely potential impacts, both positive and negative, of the decision on those with protected characteristics. The Mayor is to have due regard to the need to eliminate unlawful discrimination, harassment and victimisation as well as to advance equality of opportunity and foster good relations between people who share a protected characteristic and those who do not. This may involve, in particular, removing or minimising any disadvantage suffered by those who share a relevant protected characteristic and taking steps to meet the needs of such people. In certain circumstances compliance with the Act may involve treating people with a protected characteristic more favourably than those without it. The GLA will take appropriate steps to identify any potential negative impacts expected on those with protected characteristics.

The project is unlikely to have a direct impact on any persons whether they have any protected characteristics or not. Further consideration will be given to how the data collected could be later utilised to benefit protected groups. For example, the data collected could be used to inform individuals with accessible housing need of the available pipeline of accommodation. However much of the impact is dependent on how this data is published, and best practices will be implemented to ensure accessibility.

Other considerations

Mayoral Priorities


There are a number of the Mayoral strategies that this project seeks to support.


The draft London Plan


Chapter 12 of the draft London Plan includes the requirement for monitoring the implementation of the Plan. This currently sets out 12 Key Performance Indicators (KPIs) that will be reported on annually as part of the obligation to produce an annual monitoring report. This project seeks to automate a number of these KPIs, enable live time reporting, and support the organisation in being more agile in how it responds to and monitors policy change.


The Housing Strategy


This strategy recognises the importance of the London Development Database and commits to enhancing its capabilities. This project addresses the lack of quality information that the Housing Strategy needs to deliver its overall objectives.


Smarter London Together roadmap


One of the key objectives of this roadmap is to support the GLA in leading digital innovation, and use that role to build trust and transparency in how public data is used. This starts with the following initiatives:


  • Leadership in design and common standards to put users at the heart of what we do
  • Exploring new civic platforms to better engage citizens and communities
  • Opening up data, including real time metrics, to drive better decision making across the city
  • Joint working across public bodies to deliver digital transformation


This project addresses these objectives by standardising planning information and data. Presenting it in a standard format that will be published and analysable on a new public platform will enable communities to engage with the impacts of development in a far more effective way.


Economic Development Strategy


The strategy puts the need for data at the heart of its objectives and recognises that the availability of data will unlock informed decision making, as well as stimulate new innovative opportunities for business and services that are more responsive and inclusive to address Londoners’ needs. The strategy also envisages the need for this data as a tool to enable more joined up working between infrastructure providers.


This project provides the dataset envisaged as a starting point to deliver these objectives.


Project Costs


The projected costs for the delivery of this project in phase two are as follows:


      Staff requirements, in addition to existing resources (fixed term):


Senior Project Officer (Grade 8/9): To manage project implementation - £59,000 x 2/3 (8 months):  £39,000.


2 Delivery Officers (Grade 6/7): To work with boroughs—e.g. coordinating to update systems, agree validation list changes, etc. - £45,000 x 2 officers x 2/3 (8 months) = £60,000.


Total staff costs for Phase 2: £99,000.


The three positions to be created above will be recruited in accordance with all GLA staffing protocols.


Systems costs:


Upgrade borough systems to same level: £32,000

£1,000-10,000 per borough (as different boroughs are at different stages of readiness)

For: back office system changes, true automation with Planning Portal or equivalent.


Planning Portal changes: £25,000

For: altering fields in Planning Portal to reflect new local validations lists.


Back Office System transformation: £67,000

For: altering fields; improving ease of data extracts.


LDD Re-write: £50,000

For: completing coding on new LDD interface, including building in automation capacity.


New Hub: £100,000

For: web development to create new public hub for planning data.


Total systems costs: £274,000.


Total project costs for phase 2: £373,000


*Please note that in combination with MD2390 the total cost of this project in phases 1 and 2 is £523,000, of which £100,000 has now been secured from MHCLG.


The above services will be procured through TfL Commercial; project officers are in discussion with TfL Commercial to determine the most appropriate procurement strategies.







Risks and Issues


There are a number of key risks to the delivery of this project, these include:

Risk #

Risk description and impact

Inherent risk assessment (out of 4)

Control measures / Actions











That the submission systems may not be able to accommodate proposed changes.













  • Early conversations to ensure changes are technically feasible
  • Allocating budget for changes
  • Investigating how changes would work across all possible platforms.


That there may be a lack of LPA buy-in to project.




  • Communications strategy for boroughs, including senior political engagement, with clear narrative demonstrating business case
  • GLA commitment to do background work required on behalf of LPAs
  • Potential involvement from Central Government given national scalability


That the back-office systems may not be able to accommodate proposed changes.











  • Early conversations with providers
  • Provider-customer conversations to be convened
  • Allocating budget for changes
  • Exploration of disruptor providers, to open the market
  • Potential for GLA funds to employ legal and other services to support negotiation
  • Potential involvement from Central Government given national scalability
  • Potential solution where data circumvents back office systems


That the changes may not create savings for boroughs.









  • Potential for Cost Benefit Analysis
  • GLA budget allocated to ensure the transition does not cost boroughs
  • Potential to explore additional sources of external funding


That the funding allocated by the GLA may be insufficient to achieve objectives.






  • Budget set at level that includes sufficient contingency
  • Investigate opportunities for additional external funding


That the LDD rewrite may not be launched with automation capacity in a timely fashion.





  • Request to allocate budget internally
  • Elevating project profile internally
  • Back-up plan that allows system to go live before full GLA automation is ready


That even with changes, applicants may not input high quality data initially into applications.











  • Ensure changes are mandatory
  • Ensure different level of detail required for different level of development (home owner versus major developer)
  • Ensure boroughs continue monitoring data
  • Consult boroughs, developers, and others on the impact of changes


Financial comments

The proposed expenditure of £373,000 for phase 2 on the London Development Database and Live Data Hub will be a combination of capital and revenue expenditure scheduled for the 2019-20 financial-year.

The proposed capital expenditure of £174,000 will be funded via a draw-down from the Capital Programme Reserve.

The proposed revenue expenditure of £199,000 will funded from the Growth & Infrastructure budget (£49,000), specifically via the income received for the Infrastructure Mapping Application as approved by MD2162; and via a drawdown from the Pre-Application Planning Reserve (£150,000).

Planned delivery approach and next steps

This project will require a number of changes to the planning process. These include:

• Amending the Mayoral agreement with the leaders of London’s Local Planning Authorities regarding the data collected;
• Working with 35 LPAs to amend their Local Validation Requirements Checklist;
• Updating the Planning Portal to collect the information required in machine readable form;
• Working with back office development management system providers to enable the automated collection of information;
• Establishing reporting mechanisms to extract the data from LPAs’ back office systems;
• Updating the London Development Database to receive automated data; and
• Developing a new portal that enables live data to be published with tools to help Londoners understand the data.

A project plan has been produced to reflect this.

Activity table

This project includes eight distinct work streams that will be pursued concurrently. The anticipated timetable for delivery is as follows. Note that Phase 1 elements of the project have happened in the past, as funded by MD2390,



Delivery Start Date

Oct 2018

Workstream 1: Back Office System Market Disruption [enabling new suppliers that are responsive to LPA and GLA requirements to enter the market]

Oct 2018 – April 2019

Workstream 2: Back Office System Standardisation [ensuring borough systems are at an appropriate level of sophistication to receive automated data]

Jan – Aug 2019

Workstream 3: Submission Portal Changes

March – Oct 2019

Workstream 4: Update Local Validation Requirements

June – Nov 2019

Workstream 5: Stakeholder Communications

Oct 2018 – Feb 2020

Workstream 6: LPA Reporting

June – Nov 2019

Workstream 7: Update LDD

April – July 2019

Workstream 8: Build Data Hub

July – Nov 2019

Testing and Preparation

Oct 2019 – Jan 2020

Go Live / Delivery End Date

Feb 2020


Feb 2020

Project Closure

Feb 2020


(Please note that the above workstream timelines include procurement periods since many of the streams will require independent procurement strategies.)


Share this page