Does anyone know if the Racing Post betting forecasts are available to download ?

Question

I would like to store the Racing Post betting forecasts. Does anyone know if there is a way of downloading the betting forecasts?

Answer 1

Hi - not as far as I am aware, though it's always possible to webscrape them.

Comments

Hi Colin,

My options seem to be:

1.Cut and paste betting forecasts every day from Racing Post into excel and exporting as csv file and then loading the csv file into database.

2. web scraping the Racing Post.

With regard to this option although I am a retired IT software developer I don't have any experience of web programming.

Would it be possible to develop and run a web script that would capture all the betting forecasts every day
as well as the associated date/time and Race Course name for each forecast?

Would it need to be constantly tweaked which seems to be a disadvantage with web scraping ?

Also, I have just realised that I wouldn't be able to access historical Racing Post betting forecasts as there doesn't seem to be a way of referencing Racing Post historically - although I may be able to if I purchase a Racing Post digital subscription - it may give me access to previous issues of the Racing Post.

So I'm not sure which is the easiest option ( 1 or 2 )

Would be interested if you've got any further comments on this ?

---

Hi -

re. 1, There are options in Excel to connect to a website though you would have to manually do an import.
For (2) a completely automated option, It's possible to webscrape the Racing Post site and you don't have to be a member to access the betting forecasts.  However, I haven't seen that they are stored historically (member or otherwise), so this would be on a going forward daily basis only.  The website doesn't change often so the real risk of having to change a program that does this is slight, though as with any webscraping you have to check Ts&Cs to see it's legitimate use of the site.  Generally if you have access personally and are only scraping data for personal use (as opposed to distribution) you should be fine.

As for how to go about this, if you've never done it before it is a learning curve, but Perl, Python, Ruby and even R are all good choices for webscraping since mostly the principles are:

1.  Connect to site and loop/traverse to required pages, looking for required text as you go
2.  Use a module in one of the languages to structure the HTML (though you don't have to and can usually do everything by hand so jump straight to step 3 if too complex)
3.  Use regular expressions to extract the required text.
4.  Store it locally.  

You can do in pretty much any language though, including Java and C#.

---

Hi Colin,

Thanks for the replies to my question - because the Racing Posts forecasts are not archived it's going to be a long and tedious job of gathering the forecasts on a daily basis - probably the reason why it's not been done by anybody else.