To Do: Financial Advisor

/
Priority Project #
10 Financial Advisor #: 2
10.01

News coverage

  • Include useful market information page with podcasts from our top providers – e.g. Merryn@FT;  Prof G.  Include date and link, and summary of the content and my commentary.      Important for social media engagement 
  • Link to news scrapper
Complete
10.01

Updating Podcasts

Goal

I want to:

  • Track the top 5–10 podcasters in Personal Financial Advice
  • Pull in their latest podcast episodes automatically
  • Then manually add summaries, topics, and comments

How It Should Work

1. Podcasters (Directory)

Page: http://localhost:8000/podcast/directory/

Contains:

  • Podcaster name
  • Link to their YouTube or Apple channel

2. Episodes (Output)

Page: http://localhost:8000/podcast/episodes/index

  • This is where all scraped episodes should appear

 

What the Scraper Should Do

On the Episodes index http://localhost:8000/podcast/episodes/index, when the  “Update (scrape)” is pressed, the scraper: 

  • Goes to their YouTube channel
  • Find their Videos episodes
  • For each episode, store:
    • YouTube URL
    • Podcast title
    • Date published (open link)
    • Duration (if possible)
    • # views
  • Save the episodes into the database (avoid duplicates (don’t re-save the same episode)

What I Will Do Manually

  • Add topics
  • Write summaries
  • Add comments

Current Problem

The “Update (scrape)” button does nothing

  • No data is being pulled in
  • No error message is shown

 

Technical comment

  • I noticed if you open up a link to a video in YouTube you can see : 
    • Date of the podcast
    • # views

 

  • Does it make sense to first scrape the links from the Podcast subscriber view (eg https://www.youtube.com/channel/UCLXQalldcm6gMYMQfLMliww)   and then open each one (each an episode) to capture the  date and views data?
  • Also can we capture the text below the date/views (save in the “summary” field) and also pull in the ‘transcript’ (new field called ‘transcript’)?
  • My idea is that I could then get ChatGPT to summarise each podcast from the transcript? 

 

 

Pending
10.02

Financial advice and also generic website template

  • Create a copy of SN, and strip out all non-investment sections
  • Then copy and strip out everything except the user, CMS and basic framework
Complete
10.02

Mortgage

Background (Non-Technical Explanation)

  • A mortgage is a loan used to buy a property (house or apartment).
  • The borrower repays the loan over time with interest.

Key Concepts

1. Interest Rate

  • This is the cost of borrowing money, expressed as a percentage per year.
  • Example: 5% means you pay 5% of the loan amount per year as interest

2. Fixed vs Variable Mortgages

Fixed Rate Mortgage

  • The interest rate is locked (fixed) for a set period

Common fixed periods:

  • 2 years
  • 3 years
  • 5 years
  • 10 years

Example:
“5-year fixed at 4.5%”
→ Rate stays 4.5% for 5 years, then changes

Variable Rate Mortgage

  • The rate moves over time (can go up or down)
  • Often linked to central bank rates or market rates

3. Fees, Cashback, Legal/Valuation Frees/Other terms

Mortgages often include upfront costs:

  • Fixed fee (e.g. £999)
  • Or % of loan (e.g. 1%)
  • Cashback (essentially a negative fee)
  • Maximum Loan-to-Value (LTV)

These must be included when comparing deals.

4. Why Compare Mortgages?

The goal is to find the cheapest total cost, not just the lowest rate.

 

======================================================================

2. “Swap Rates?” (Important Context)

  • Swap rates are market interest rates used by banks.
  • They represent the true cost of fixed-rate lending in financial markets.

Why They Matter

Banks price mortgages based on swap rates

If:

5-year swap = 3%

Mortgage = 5%
👉 Bank margin ≈ 2%

This allows us to:

  • Compare banks
  • Identify overpriced or cheap mortgages

3. System Objective

Build a system that:

  • Collects mortgage offers from aggregators (e.g. MoneySuperMarket)
  • Stores them in a structured format
  • Collects daily swap rates
  • Enables mathematical comparison of mortgage deals vs market rates

 

CODING TO DO

4. Mortgage Rates Scraper

See: http://localhost:8000/mortgage/rates/

This page contains buttons:

  • 2-year
  • 3-year
  • 5-year
  • 10-year

Each button leads to a page showing mortgage offers for that maturity term

What to Scrape (Per Mortgage Product)

For each mortgage on the page, extract:

  • Link on moneysupermarket
  • Bank Name
  • Term (ie fixed period in years (eg 2yr, 3yr, 5yr or 10yr)
  • Interest Rate (%)
  • Rate Type 
  • Fees
  1. Fixed amount (£)
  2. Percentage (%) if available
  3. Cashback
  • Maximum LTV
  • Free Legals 
  • Free Valuation

Storage

Create a new record via:  http://localhost:8000/mortgage/rates/new

Scraper Logic

For each term (2, 3, 5, 10):

  • Open the page (2yr, 3yr, 5yr, 10yr)
  • Loop through all mortgage deals available
  • Extract required fields
  • Save one record per mortgage deal available

Maintaining a record over time…  (let's tackle handling versioning over time after we have the scrape working)

  • If the Mortgage Deal already exists in the database (same Lender, same maturity term, same rate, same fees, status=Active) then don't create a new one record
  • If it is a Mortgage Deal, set “DateLaunched” as today and set status as Active
  • If a mortgage record (Active) is no longer present, mark the date "DateWithdrawn" as today

 

5. Swap Rate Scraper Specification

See:  http://localhost:8000/swap/rates/

What to Extract

For each tenor:

  • 2-year
  • 3-year
  • 5-year
  • 10-year

Extract:

  • Rate (%)
  • Tenor
  • Date (today’s date)

Execution

Run once per day (cron job) 

6. End Goal (Why This Matters)

This system allows  comparison of mortgage deals on a like-for-like basis.  I can explain why to you, but not important for the coding exercise

 

Pending
10.03

Updating contributor site

  • Rich text editor for the Summary and Comments fields
Complete