MD2390 London Development Database Automation & Live Data Hub project

Type of decision: 
Mayoral decision
Date signed: 
16 November 2018
Decision by: 
Sadiq Khan, Mayor of London

Executive summary

This Mayoral Decision (MD) seeks approval to fund the development of an up-to-date, automated data collection and monitoring system for the London Plan. This project should: enable a robust evidence base on which to prepare new planning policies and make decisions on planning applications; reduce the cost of collecting development data for Londoners and local planning authorities; and provide a clearer, more transparent picture of what is happening on the ground for Londoners, SMEs, Government, and others interested in London’s growth.


That the Mayor approves:

The allocation and expenditure of up to £150,000 on phase one of the project to deliver an updated, automated London Development Database and related improvements.

Part 1: Non-confidential facts and advice

Introduction and background

The Greater London Authority Act 1999 places responsibility for strategic planning in London on the Mayor and requires him to produce a strategic spatial strategy for London, which he is also required to keep under review.

Since 2004 the effectiveness of that spatial strategy has been monitored through the London Development Database and reported through an annual monitoring report.

The way this works has broadly not changed since 2004 and requires each of the planning authorities in London to review each permission they grant and report permissions for developments which:

• Create or lose 1 or more dwellings;
• Result in a change of use of more than 1000m2; or
• Results in the loss or gain of public open space.

This data is collected and entered into the current London Development Database through a web portal, often manually, and checked by GLA officers.

The data is collected relating to:

• When permission is granted;
• When the development commences; and
• When the development is completed.

Data is also collected on when the planning permission expires unimplemented.

Whilst still the only database of its kind, and ground breaking for its time, technology, the planning process, and the development industry have since moved on not least in the expectations that data will be collected to enable the effectiveness of policies and development on the ground to be assessed much more comprehensively than the limited dataset above.

The Mayor’s London Plan is soon to be adopted and the limitations of the current London Development Database (LDD) mean that the effectiveness of this critical policy document on non-referable schemes will not be measurable for over a year, if not 18 months, after the policy document has been agreed and published.

The GLA has other projects that rely on the LDD for their effectiveness, in particular the London Infrastructure Mapping Application, the soon to be launched accessible housing register, and the work of the City Intelligence team on demographics, schools, and other social infrastructure. The LDD’s current limitations constrain the effectiveness of these tools.

Specialist services are required therefore, to enable the LDD to be updated to meet the GLA’s London Plan needs efficiently.

The update of the London Development Database will address these issues by automating the collection of monitoring data for all planning application types.

This request for a Mayoral decision aligns with the Smarter London Together roadmap and the GLA’s wider work stream regarding Digital Planning. The project will be delivered to meet the criteria of the Local Digital Declaration of which the GLA is a signatory.

This request for a Mayoral decision is also informed by a discovery exercise where GLA staff visited 26 Local Planning Authorities (LPAs) in early 2018 to identify how they currently complete LDD requirements; their existing back office systems and management practices; the cost of the status quo; the quality of the data they provide; and issues they face completing the requirements. A summary of this research can be found at

Objectives and expected outcomes

The objectives of the automation of the London Development Database are:

• To have a clear, up to date and robust evidence base on which to prepare new planning policies and make decisions on planning applications;
• Reduce the cost of collecting development data for Londoners and local planning authorities; and
• Provide a clearer, more transparent picture of what is happening on the ground for Londoners, SMEs, Government, and others interested in London’s growth.

The outcomes of this project are:

• A central register of all planning applications submitted within the GLA area with key monitoring data in machine-readable format; and
• A dataset that will inform decisions, with published tools to enable Londoners to analyse the data.

There are likely to be other outcomes from the delivery of this automation, including:

• E-Alert notifications for Londoners of all developments that might affect them, regardless of borough boundaries, enabling Boroughs to move away from written neighbour notification letters;
• Opportunities for business to innovate based on the live data feed of information, which we hope will stimulate new markets and opportunities for business;
• Enable smoother delivery of housing; and
• Encourage boroughs to use the connected levels of information to undertake more specific monitoring of developments and generate faster payment of CIL and policing of building regulations.

This MD funds phase one of this project, to be completed by 31 March 2019. The above outcomes would be delivered at the close of phase two in February 2020 (contingent on an additional MD).

Equality comments

The public-sector equality duty (PSED) under section 149 of the Equality Act 2010 requires the identification and evaluation of the likely potential impacts, both positive and negative, of the decision on those with protected characteristics. The Mayor is to have due regard to the need to eliminate unlawful discrimination, harassment and victimisation as well as to advance equality of opportunity and foster good relations between people who share a protected characteristic and those who do not. This may involve, in particular, removing or minimising any disadvantage suffered by those who share a relevant protected characteristic and taking steps to meet the needs of such people. In certain circumstances compliance with the Act may involve treating people with a protected characteristic more favourably than those without it. The GLA will take appropriate steps to identify any potential negative impacts expected on those with protected characteristics. The project is unlikely to have a direct impact on any persons whether they have any protected characteristics or not. That being said further consideration will need to be given to how the data collected is used and how that may have an impact on any protected group. For example, the data collected to inform individuals with accessible housing need of the available pipeline of accommodation could have a positive impact. However much of the impact is dependent on how this data is published.

Other considerations

Mayoral Priorities

This project contributes to a number of the Mayor’s priorities, including:

  • Enabling accurate data on delivery, pipeline and decisions for the provision of housing for Londoners and genuinely affordable housing:


‘My first priority will be tackling the housing crisis. We need to build more homes, including more genuinely affordable homes for Londoners, and fewer gold bricks for overseas investors.' (Manifesto);


  • Enabling joined up decisions for infrastructure planning—helping utilities plan for growth so that lack of infrastructure does not become a blocker to housing delivery—as well as providing opportunities for business to use and commercialise this information to further opportunities for Londoners:


“When it comes to planning London’s future economic development, infrastructure, skills and housing will be my foremost priorities” ; “The next Mayor must start planning and delivering the infrastructure and new capacity for the future straight away…’ (Manifesto);



  • Providing a collaborative, cross-London effort toward open data and transparency:


‘Put an open data strategy at the heart of London government, with a new London data office working to bring data from across London’s boroughs and public agencies together, and opening it up to enable quicker decision making, better services, more efficient government, and greater transparency.’ (Manifesto); and


  • Enabling centralised collection of this data will enable the delivery of more joined up services for all, removing the postcode lottery of planning services, enabling Londoners to become more engaged in shaping the environment they live in.


Project Costs

The protected costs for the delivery of this project in phase one are as follows:


      Staff requirements, in addition to existing resources, through 31 March 2019:


Senior Project Officer (Grade 8/9): To manage project implementation  - £60,000 x 1/3 (4 months) = 20,000


2 Delivery Officers (Grade 6/7): To work with boroughs—e.g. coordinating to update systems, agree validation list changes, etc. - £45,000 x 2 officers x 1/3 (4 months) = £30,000


Total staff costs: £50,000


Systems costs:

Upgrade borough systems to same level: £42,000

£1,000-10,000 per borough

For: back office system changes, true automation with Planning Portal or equivalent


Planning Portal changes: £25,000

For: altering fields in Planning Portal to reflect new local validations lists


Back Office System transformation: £33,000

For: altering fields; improving ease of data extracts


Total systems costs: £100,000


Total phase one project costs: £150,000


The above services will be procured through TfL Commercial; project officers are in discussion with TfL Commercial to determine the most appropriate procurement strategies.

The three positions to be created above will be recruited in accordance with all GLA staffing protocols.


Risks and Issues


There are a number of key risks to the delivery of this project, these include


Risk #

Risk description and impact

Inherent risk assessment (out of 4)

Control measures / Actions






That the submission systems may not be able to accommodate proposed changes




~early conversations to ensure changes are technically feasible
~allocating budget for changes
~investigating how changes would work across all possible platforms.


That there may be a lack of LPA buy-in to project




~communications strategy for boroughs with clear narrative demonstrating business case
~GLA commitment to do background work required on behalf of LPAs

~potential involvement from Central Government given national scalability


That the back office systems may not be able to accommodate proposed changes




~early conversations with providers
~provider-customer conversations to be convened
~allocating budget for changes
~exploration of disruptor providers, to open the market
~potential for GLA funds to employ legal and other services to support negotiation

~potential involvement from Central Government given national scalability

~potential solution where data circumvents back office systems


That the changes may not create savings for boroughs




~potential for CBA
~GLA budget allocated to ensure the transition does not cost boroughs

~potential to explore additional sources of external funding


That the funding allocated by the GLA may be  insufficient to achieve objectives




~budget set at level that includes sufficient contingency
~investigate opportunities for external funding


That the LDD rewrite may not be launched with automation capacity in a timely fashion




~request to allocate budget internally
~elevating project profile internally

~back-up plan that allows system to go live before full GLA automation is ready


That even with changes, applicants may not input high quality data initially into applications




~ensure changes are mandatory
~ensure different level of detail required for different level of development (home owner versus major developer)
~ensure boroughs continue monitoring data


Financial comments

The proposed expenditure of up to £150,000 on the London Development Database and Live Data Hub during phase one of the project will be a combination of capital and revenue expenditure in 2018-19. The estimated expenditure budget and proposed funding sources is summarised (*on the PDF decision document).

The proposed capital expenditure of £100,000 scheduled to be incurred in 2018-19 will be funded by a revenue contribution to capital from existing revenue budgets, specifically via Planning budgets held within the Development, Enterprise & Environment Directorate (DEE) and a drawdown from the pre-application planning reserve as summarised (*on the PDF decision document).

The proposed revenue expenditure of up to £50,000 will funded from the Growth & Infrastructure budget, specially via the income received for the Infrastructure Mapping Application as approved by MD2162.

It should be noted that phase 2 of the project is still being developed and subject to funding being made available will go through the Authority’s decision-making process once fully scoped out.

Planned delivery approach and next steps

This project will require a number of changes to the planning process. These include:

• Amending the Mayoral agreement with the leaders of London’s Local Planning Authorities (LPAs) regarding the data collected;
• Working with 35 LPAs to amend their Local Validation Requirements Checklist;
• Updating the Planning Portal to collect the information required in machine readable form;
• Working with back office development management system providers to enable the automated collection of information;
• Establishing reporting mechanisms to extract the data from LPAs’ back office systems;
• Updating the London Development Database to receive automated data; and
• Developing a new portal that enables live data to be published with tools to help Londoners understand the data.

A project plan has been produced to reflect this.

This project includes eight distinct work streams that will be pursued concurrently. The anticipated timetable for delivery is as follows. Please note that the workstreams with dates in bold will commence during phase one of the project covered by this MD:

Activity table



Delivery Start Date

Oct 2018

Workstream 1: Back Office System Market Disruption [enabling new suppliers that are responsive to LPA and GLA requirements to enter the market]

Oct – Nov 2018

Workstream 2: Back Office System Standardisation [ensuring borough systems are at an appropriate level of sophistication to receive automated data]

Oct 2018 – April 2019

Workstream 3: Submission Portal Changes

Oct 2018 – March 2019

Workstream 4: Update Local Validation Requirements

Oct 2018 – April 2019

Workstream 5: Stakeholder Communications

Oct 2018 – Feb 2020

Workstream 6: LPA Reporting

Jan – Aug 2019

Workstream 7: Update LDD

April 2019 – June 2019

Workstream 8: Build Data Hub

March – Sep 2019

Testing and Preparation

Sep 2019 – Jan 2020

Go Live / Delivery End Date

Feb 2020


Feb 2020

Project Closure

Feb 2020

Share this page