2014 in review

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here’s an excerpt:

A New York City subway train holds 1,200 people. This blog was viewed about 5,000 times in 2014. If it were a NYC subway train, it would take about 4 trips to carry that many people.

Click here to see the complete report.

Advertisements

DECC and why Government Computer Projects Fail

Back in the summer I bid for the tender to provide the self same Renewables Map facility but paid for by DECC.  If you read the specification within the tender document it all but exactly describes Renewables Map. There are a few more reports to add, but this is a very small proportion of the task. The biggest challenge is getting the data. More than that, DECC used to refer to Renewabales Map as if it was their own, then, in 2011,  changed the name of their resource to renewables map – a bit confusing?  Praise or plagiarism? .

Anyway, having been dismissed out of hand I wrote to my MP, got a reply through him from Ed Davey – a standard “We’re the Government, we always do the right thing” letter.

Then wrote to  Jane Redrup in DECC who replied in depth.  Thanks Jane, but if this is an indication of how DECC select providers, God help us.

I have copied the response below, You can also download here: DECC / Jane Redrup’s response 

But to summarise, this is the gist of the response: “We are not at all interested in what you have done or can show you can do, we want to see a jolly good essay.  All marks towards whether you get the tender bid will be based on grammar, spelling and punctuation.  Extra marks got to people who make claims about what they will do. Being able to show you are already doing exactly what we are asking for will not count in your favour”

Unbelievable!  But if this is an indication of how Government departments select suppliers, then we are truly doomed!

As to the winning bidder.  This is a company called Eunomia.  They are now one of my larger users.  See here:  Eunomia Research and Consulting use of Renewables Map.  Note that this (can I call it plagiarism? not sure) activity started immediately after they won the tender, clearly they had no interest nor experience in this subject until they got the contract. I expect they wrote good essay’s though!

Obviously to get the most of Jane Redrup’s response would need an understanding of the tender document, in particular the division between the delivery of REDP (Renewable Energy Database Project) and RESTATS (Renewable Energy Statistics)  I bid for REPD, not RESTATS Jane jumps between the two in critiquing my bid, also the response is inconsistent and with knowledge of tender / bid and evidence, certainly confused. NB. I did NOT bid for the RESTATS tender, I bid for the REPD tender. However I am being judged based on not proposing to deliver the RESTATS requirements!

Here’s an example from the initial dismissal :

The Sift Panel commented that the bid demonstrated experience of project management, with good evidence of data collection and development. However there was little evidence of how that would be applied to this work area. The bid also failed to demonstrate a good understanding of renewable energy and policy issues and of DECC’s requirement.”

However there was little evidence of how that would be applied to this work area. ”  Urm… Maybe the evidence of what is now accepted as the best collection of renewable energy project data in the UK? Do they have so little knowledge of what goes on beyond their office?

And with the experience of Project Management.  Actually I gave examples of successful projects, such as leading and largely doing a study of the Supreme Court in Pakistan that resulted in a successful bid for £millions from the ADB. etc etc.. so clearly recognised.  I didn’t say experience and training in PRINCE and PRISM. How can these be applied to what is in essence a process rather than change?

Anyway, have a look at the scores further down. Based on what Jane Redrup has said these are based on what I described not on evidence. So, however brilliant I am at developing databases, integrating GIS, automatic downloads of metoffice data, pick and point map entry, rdbms etc etc unless I can write a nice essay describing it, ,,,,  well, words fail me.  No wonder the Government’s IT record is so awful when this is the way tender bids are decided!

Letter text below
Dear Mr Mallett

Renewable Energy Planning Database Procurement Process

Thank you for your email of 28 October 2014, in which you seek further feedback about your tender to ‘Maintain and publish a planning database for renewable energy installations’ (Tender reference 830/06/2014).

I will address each of your points in turn but more generally, I would like to reiterate that DECC takes an evidence-based approach to procurement in the interests of fairness to all bidders. It is the Tenderer’s responsibility to ensure that the information contained within the response to the Invitation to Tender (ITT) meets all the requirements of the ITT on a full and comprehensive basis.

This procurement was for the maintenance and publication of a planning database for renewable energy installations. The ITT set out at Section 12 the scope of the work and the expectations of what the contractor would be required to do, and at Section 13, the requirements and outputs.

In particular paragraph 13.1 set out that the ‘overall aim of the work of the planning database is to ensure that Government can continue to have access to accurate data on the deployment of renewable electricity projects as they pass through the planning system. The emphasis will be on the capture of accurate and timely data on proposed and existing planning applications’.

Section 13 also listed specific objectives and advised that the tender should set out the proposed arrangements for covering these objectives.

To enable the Evaluation Panel to assess the best bid, the ITT set out at Paragraph 26.1 criteria for assessing the tenders, and the weighting to be applied. This included wording such as ‘experience / ability to demonstrate…’.
As part of this assessment, the Evaluation Panel was looking for bidders to demonstrate clearly and convincingly how they met the criteria and therefore the requirements of the ITT.

In submitting your Tender, you referenced a published renewables map as evidence of your ability to meet the requirements of the ITT. However, it was still necessary to support this by explaining in the main body of the bid how and
why it would meet all the requirements of the ITT (such as consistency, quality, accuracy, timeliness) and to provide assurance on how the outputs would be met to the required level. Unfortunately, there was insufficient detail in the bid
for the Evaluation Panel to verify this.

Simply put, you would have scored higher marks had you:

  • Provided supporting evidence within the body of the bid on how your renewables map met the requirements of the ITT; and
  • Addressed areas outwith the scope of the map in sufficient detail to provide evidence or demonstrate an understanding of the requirements.

For example, how the map and your proposed data collection processes were sufficient to meet DECC’s needs and quality standards; your approach to data quality assurance.

As requested, I will now address the individual points raised in your email. For ease of reference I have replicated your email below, with my response added in red below each comment.

Simon Mallet email:  (and response from Jane Redrup highlighted) 
I tendered for the REPD part of the overall RESTATS / REPD project and was dismissed at an early stage. The general statement was:

“However there was little evidence of how that would be applied to this work area. The bid also failed to demonstrate a good understanding of renewable energy and policy issues and of DECC’s requirement.”

Looking at the first sentence “however there was little evidence of how that would be applied to this work area” the specification of what was actually required as an output in every sense described renewables map. A resource that I have developed and axpanded over the years, even so far as to change the original restats target from sub 1MW to anything over 1MW.  The statement is irrational and ignores the evidence presented. Simply the fact that I was (as stated within the bid) and still am already doing what was required as the major output of the tender (and actually doing it better. The additional reporting was just a case of formatting the collected data in a different way.

Although you mention you would work with others, we would have liked detail on how you would do this, how you would guarantee the quality of the work and how you would meet DECC’s requirements.

The tender makes only very high-level, general references to working with developers and local authorities. We are aware of the difficulties of sometimes achieving timely and accurate data collection, so would have expected your response to include detail as to how this would be achieved or how the risk of a lack of engagement from them would be managed.

(NB.  I don’t know what statement Jane is responding to, its not at all related to my statement / question. As to dealing with developers and local authorities? Well, I’ve been accessing that data successfully for at least 6 years, many real projects I detail are not yet included on REPD) 

Going on from there: “The bid also failed to demonstrate a good understanding of renewable energy and policy issues and of DECC’s requirement” Having developed and managed the renewables map for over 6 years to the point where it is a resource used more than the equivalent part of DECC’s equivalent by 2 orders of magnitude, again this is an irrational statement.

As mentioned above, your tender concentrated on the publication of your map. You would have received higher marks had you, for example demonstrated an understanding of how Government policy on renewables has evolved over time, and mentioned the current challenges the REPD is designed to help address. For example, the management of financial incentives schemes and the need to meet targets and manage budgets.

I would also draw your attention to the statements from NAREC and STFC. You have seen these already.

Extracts and second-hand statements from third parties are not relevant unless they demonstrate meeting the requirements.

My contention is that as an individual, even though I was able to present proof of my ability to satisfy every requirement within the tender, my bid was always going to be dismissed.

As explained by Ed Davey in his response to the Rt Hon Sir Hugh Robertson, each proposal was assessed by an evaluation panel and marked in accordance with the published evaluation criteria and weighting, set out in the Invitation to Tender. The three suppliers with the highest scores from the written proposal were then shortlisted and invited for interview, two of which were SMEs.

The process was carried out in a fair and transparent way, overseen by a Senior Procurement manager. Whether the proposal was submitted by an individual, SME or large organisation was immaterial as the assessment depended on the evidence submitted within the tender.

I would also like a response to:

For about 2 years, renewables map was referenced by DECC / RESTATS as the definitive resource for renewable energy projects. Please can you clarify how this fits with the above dismissal of my ability to provide this resource formally. I can only think that you are happy to have it for free, but, for some reason will not entertain a bid to supply what was previously used by you for free I simply cannot understand this, unless I consider less salubrious reasons.

As set out above, the Panel could only consider the evidence submitted. I cannot comment on the previous reference or use of renewables map. However as one of the primary users of the REPD over the past 4 years, I can confirm that the
REPD is a key component of DECC policy making. Other published databases are unlikely to be used for DECC purposes as they are unable to provide the same level of certainty over quality, accuracy and timeliness of data.

(The concern here is that just because REPD is stated as a Govt resource it is by definition accurate and up to date. Having spoken to the previous provider of the data for RESTATS/REPD, they admitted that mine “Renewables Map” was more accurate and more up to date, the fact that the new provider is now data mining Renewables Map rather confirms this, so a comment to Jane, your statement is both arrogant and stupid! )  

I have stated that renewables map gets 60 times more visitors than the equivalent part of the restats/repd web presence. This is based on the data provided within the FAQs of the tender. Can you clarify web page views and unique visitors with a clearly stated time period, that information was omitted from the FAQs and cannot/ could not be considered ‘commercial in confidence;..

The contractor has advised that the data reported related to June 2014 and was intended to give a representation of the traffic levels.

Also, and perhaps as important as all the above. Following Eunomia’s award of the contract they are rapidly becoming the most frequent users of renewables map. If they are so much better that they justify being paid for this data, why are they
constantly data-mining my website. I cannot think of the words to express how contemptible that is.

This is not for DECC to respond.

I would appreciate a clear and honest response – please include my MP (via Stainton James) in your reply. I hope you will give him the respect, in your clear and full response to all of the above, that we both deserve.

deccscores

Developments to Renewables Map

Apart from the data being the most complete and accurate of any of the equivalent resources, Renewables Map is being further developed towards being a one stop shop for all data that is generally available online.

Showing More data

Many of the over 3,000 projects listed within renewables map now have the DECC and for active projects, the OFGEM data shown. This data is of course only available where the corresponding data has been listed by these organisations.  OFGEM is invariably up to date as subsidy payments are dependent on OFGEM being informed. DECC is not so consistent in maintaining its data, don’t blame me if that data is missing.

Address data is being added along with a separate postcode and county entry.

(ongoing) Associated company data is now being applied to all projects showing developer, owner and operator.  This is intelligently linked so by clicking on the associated business you get to see their contact details, then a list of their projects, again all hyperlinked.

(coming soon) The ability to select data sets and download to a csv file. An example might be:

All solar projects in Kent with number of panels, acreage, address, developer, capacity.

or

The latitude and longitude and elevation of all Vestas V90 turbines in the UK along with the associated project.

Offshore projects

For offshore projects, there is a direct link to the 4coffshore resource and their map embedded (with 4coffshore’s permission) .  Have a look at Kentish Flats Wind Farm  you will also be able to see corresponding data from DECC and OFGEM where this exists.

The MAP

The ability to specify in greater detail what will be seen, eg just particular turbines or all projects for a particular developer or in a particular county.

Smart Data 

In the future will be the ability to login and manage private data alongside data from the Renewables Map.  That is fairly straightforward as the user management exists within the software and I use it, however it will have to pay for itself first.

But I’m not seeing all this extra data !

Your 1st visit of the day will show most of the additional data, then you will be switched back to the restricted version.

Alternatively have a look at   http://www.renewables-map.co.uk/details.asp?pageid=1940

Renewables Map gets over 10,000 unique visitors each month looking at over 30,000 project pages. 30% are regular visitors, many using the renewable energy  map as a routine resource.

My aim was to have everything on the renewable energy database and map a completely free resource and have been writing to the biggest users who are also the biggest power companies and generators.

I’ll list the biggest power companies: EDF; E.on; npower; Scottish Power;SSE . Lets add the generators: Airtricity, Ecotricity; Infinis; Energis; REG; RES; lightsource;  – oh, and every other generator I’ve missed (sorry not too list you) . Lets add the wind companies like Siemens; Vestas; Enercon; Repower and Vattenfall and not to forget the associated companies like National Grid,  Narec, STFC.  Actually, find a list of all companies in any way associated with energy and specifically renewable energy  and they are my biggest users. Don’t forget to add the BBC and Associated Press – actually all the media.  Then there is every single council and Government Department including Parliament and even the FCO!

Why list all of these companies and organisations?   Well, I had started to write to them, they use the resource and it would be so much easier to sponsor than have a paid for resource. That at least would make it accessible to the equally large number of schools, colleges and universities that use the data.

So, what sort of response? Well, RWE and Vattenfall replied – the only ones. Both were a NO, they remain major users.

Everybody else didn’t even bother replying – but I’ve only contacted a small number so far. It gets very depressing.  Ecotricity emailed to ask for a download of all the data I have collected over the last many years.  I politely suggested that I couldn’t do this for free, I didn’t even get a response, they remain major users, I’ve tried contacting again, no response.

Same for Bloomberg – have you seen the price of their terminals !

Crown Estates – they even state they offer sponsorship – No!

You can read about my attempt to simply carry on doing this but for DECC here: DECC RESTATS .  depressing isn’t it.  The people who won the contract – Eunomia rapidly became one of the biggest visitors to Renewables Map, only stopping when I wrote to Jayne Redrup of DECC complaining about what clearly looked like data mining /  plagiarism.

Anyway a last attempt at keeping all this free is going to be an approach to Heritage Lottery fund. I really want this to remain the best and be free access. Its just that as an individual providing a service to – yes the list of companies and organisations above –  that is a bit silly.

So.  renewables map will aim go from strength to strength, remaining the best , it will unfortunately move towards a paid for model.

Sorry

Comparison between RESTATS and Renewables Map

With the tender offer for the provision of the Renewable Energy Project Database, they gave out stats about the usage of their system. Well, if you can make head or tail of them, then you are better (wo)man than I am (I’m being PC). If anything it shows a poor approach to the collection and promotion of data.  Not giving a time-frame makes the data all but meaningless.

See DECC / AEA Ricardo’s statistics

RESTATS (then run by AEA Ricardo, now run by Eunomia) quote for “Renewable Energy Maps” is 385 page views.

They do not say whether this is per hour, day, week or month. Elsewhere within an answers to questions, they are using per month figures, so I  guess this is per month, but it might not be.

Renewables Map (www.renewables-map.co.uk) on an average day gets over 1,000 page views, from 300 to 350 visitors,

Over an average month, there are about 7,000 distinct visitors, with 25,000 page views.

(from October 2014 this has increased to 10,000 unique visitors looking at 30,000 pages)

I am excluding Search Engines updating their database, these are real people looking at real data.

So, for Map / project enquiries, “Renewables Map” gets between 60 to 70 times more (from October 2014, a good 100 times more) use than the official Government version.

Looking at stats for ALL pages on the RESTATS website, they quote  11,890 page views, which rather reinforces this as a per month value.  (Google Webmasters analysis says that Renewables Map gets visited by 10% of ALL people doing searches for this type of renewable energy information).

Going back to the 11,890 page views – that’s for all parts of DECC’s presentation of information about renewable energy projects. Renewables Map gets over 25,000 page views!

It would be great if Eunomia (who are regular visitors to Renewables map, maybe this is where they get their data??) could give some usable stats on the use of the RESTATS site. I think it will be a bit too depressing to make it public?

WInd Farm Planning Tool

Adding data to the map, in particular when it comes to individual turbines has to be relatively quick and easy.  Actually adding turbines can be  quite cathartic, especially where the locations are easy. I’m adding the 40 or so turbines proposed for Strathy South in between writing this.

Proposed Strathy South Wind Turbines

It is a very simple point and click operation with the turbine data coming from the base project information, adding them to the map includes elevation and latitude / longitude along with the ability to modify the individual turbine in detail. ie it can be one of many all the same or unique. Projects and individual turbines are cross-referenced to a standard list of wind turbines each with their characteristics, so, rather than having to add to much detail, simply stating the turbine make and model is the cross reference to start, optimum and stop speeds along with data showing capacity and output based on wind speed.

Showing Calculated Wind Farm Output

Once the turbines are added, the map can then show calculated output for the project based on wind speed, actual and forecast that is downloaded routinely from the met office, in fact there is a growing history of wind strengths and direction for the UK that can be applied to the wind projects.

Test wind speeds can also be  applied to a wind farm.

Little Cheyne Court Wind Farm with forecast output and output test based on input wind speed

While I don’t have access to real output based on wind speed, the software to develop a balancing factor based on real output taking into account turbine characteristics, elevation, wind speed and direction would be fairly straightforward.

Wind Farm Forecasting Tool

Which means that it would be equally straightforward to develop a virtual wind farm with any number and type of turbine, then apply test wind speeds giving a realistic view of what the output might be throughout the year.

The additional benefit would be the “virtual” wind farm can be seen in the context of existing wind farms close by. Any dwellings, roads and other structures can be seen, and using either or all of OS maps, Google maps and satellite views, the impact can be easily seen.

If this is a wind farm that is going to go into the “proposed” or planning stage, the farm can be easily publicised showing the real proposal rather than a variation proposed by the anti lobby gaining credence.

I haven’t got to the point of showing the wind farms and turbines as an elevation with the correct sizes, but again, that is just a matter of coding, the location and elevation data is stored.

How easy would this be and how would it work in practice?

At the time of writing, the detail has to be developed, however the hard part of the software is complete and used routinely.

Screen shot of turbine entry routine: point ;add; save; point; add; save

Screen shot of turbine entry routine: add; point ;save;  add;point; save

I would see a relatively simple routine whereby a developer using the renewables map in the manner proposed above would have their own password protected login.

They would then be able to create projects in exactly the same way as renewable energy projects that are in the public domain are created, however they would be tagged as private to that login, maybe access limited to an IP?

The created project would then exist within the database under the control of the user, it might be left as a private project, modified so it was made public and part of the map or maybe public but only through a particular URL? Maybe deleted.

How much would this sort of thing cost?

Judging by other projects I have been involved in, ask for a quote from a consultant for much the same, then take a zero off!

Please feel free to contact me about this using one of the contact forms on the map.

Atlantic Array Cancelled

The BBC’s online article “Offshore wind farms face uncertain future” 

Atlantic Array’s 1.2GW was part of 32GW of planned offshore wind capacity

 

The article looks at the viability of offshore wind farms. While these are clearly more expensive to install and maintain, the challenges being not just the distance from port and dealing with the weather, but also establishing electrical connections, undersea cable and a link into the onshore grid, then there is the salt! Conversely, the wind is more consistent and with bigger turbines, far more capacity can be installed.

One of RWEs comments was that the project might be viable with the next round of technology, I guess both bigger turbines going up to 5MW capacity and maybe floating turbines that in extremis can come back to port to be maintained.

The idea of a tethered floating turbine, or maybe on jack-up type bases,  using oil industry type technology must be an attractive alternative to static pin piled turbines? The advantages, with a working system would circumvent many of the current disadvantages. Both jack-up and floating/tethered can be fully fitted out in dock and potentially returned to dock for major refurbishment on a routine basis. Rather than having an installed turbine that degrades over the years to the point where it is fit for the scrap yard, along with the huge cost of dismantling, a portable wind turbine might go on for many decades.

In the case of non fixed turbines, I would tend towards vertical axis turbines. With a horizontal axis turbine there is always going to be a stress as the wind tends to push the turbine mountings over.  A VAWT in spinning on a vertical axis will have a gyroscopic effect, aiding the stability of the installation. Agreed, the technology is in a sense still developing, but surely the ultimate benefits will seriously outweigh that additional cost of development, certainly more so than the current tendency towards what seems to me the brute force approach of ever bigger turbine blades with the concomitant problems of more bearing problems, bird strike, bigger foundations.   

Midnight Sun – Solar Energy

I’ve just seen an advert for “Midnight Sun” in the daily mail. This is from a company called AWE – nothing to do with AWE Aldermaston which makes nuclear weapons.
Along with solar panels and the rest of the eco type stuff, they are marketing a gizmo called Midnight Sun which supposedly stores energy generated through the day for use during the night, thus saving you money ! Ostensibly a good idea but what is it.
Well, it can only be a battery, a battery charger and an inverter, hopefully a bit more electronics to synchronize with the phase of any solar panel inverter (Otherwise you are simply charging from the mains) .
Further research says about 9kWh of storage and 5kWh available (think 9 and 5 units of electricity ) So in simple terms a deep cycle battery, on ebay look for deep cycle gel battery 12volt  170ah which equates to about 8.5kWh and is about £250.  Battery chargers aren’t expensive and inverters much the same, lets say £100 each for the top of the range, maybe £600 worth of kit including something to balance the phases.  The deep cycle battery has maybe 700 cycles before it needs replacing.
This £600 worth of equipment will potentially store 5kWh of unused solar energy, assuming that over the day there is any unused solar energy and that the sun is out, so it might work at near capacity during the summer , hardly at all during the winter.
So we might save at max 5 units of electricity, lets say an average of 3 units a night -I’m being very generous!  Each unit represents maybe 6p taking into account the savings in not buying it and the loss in not exporting it. Our £600 investment  saves us at best about 20p a night or maybe £60 a year!  So, ignoring battery replacement and any “money” cost associated with having £600 invested in batteries and not invested in savings, even at 1%, we have at minimum a 10 year payback.
Now have a look at their website : http://www.aweenergy.com/midnight_sun.php.
I was drawn to this by the full page advert in the daily mail for “Midnight Sun”  / “Solar PV Payback Scheme” It sounds good, no mention of batteries, lots of “Solar Energy” , “Stored energy” etc. No mention of price nor actual savings.
Now read this: http://www.navitron.org.uk/forum/index.php?PHPSESSID=c24b607093cc3df8638529bcb5fefcbb&topic=18595.15 and we get an idea of the price, not £600, not £1,000 which might allow a bit of profit.
No, to get a price, apparently (according to the Navitron subscribers) you have to have the salesman visit, salesman usually equals a lot of commission .  Well, the prices I have come across are from £5,500 upwards.
So, with a saving of £60 a year and that assumes the system really stores ONLY excess solar power! That means over 100 years to recoup your investment, but the batteries won’t last that long, maybe 5 years?
I suppose as a UPS / Battery backup it might work, but UPS’s are in the region of £100 not £1,000s