Thursday, July 27, 2023

Re: Routing multiple IPv4 blocks

Hello,

> it is up for debate, their newer LNS are still rather crashy, it's
> fairly easy to trigger their "portsuspended" thing and knock users
> offline, and their peering/transit ports are relatively low capacity
> compared to many ISPs so they're more easily affected by DoS.

I believe it is because they are trying to make their own hardware
(firebrick.uk) instead of using already established enterprise level
gear which is used by other ISPs, notibly BT were the idiots who trusted
the Chinese for their gear (huawei), unfortunately I use an old
Openreach DSL modem, which was produced by huawei, no way of avoiding
them unfortunately (well cheaply, I could always use a router + modem
combo in modem mode, but I rather a dumb modem).

To be honest, I am not against companies trying to do their own thing,
and I assume the newer LNS problems will slowly be ironed out.
Personally I have not had any issues with my service, in fact I have
found it incredibly reliable, and ridiculously enough I get much better
latency over A&A than the other line provided by virgin media (coaxial),
but virgin media has always been pathetic, fast speeds, low reliability,
and anyone who wants to say virgin media is reliable, take a look at
downdetector and their twitter, they have had severe problems recently.

> imho the main reason for paying their premium is if you specifically
> need features which they offer that others don't (e.g. the multi line
> balancing/fallback options can be handy sometimes, and they're one of
> a small handful of UK ISPs who will do "bring your own provider
> independent IP block and we will route it").

I love all the features, unfortunately I do not have the money to mess
with that cool stuff, but I would if I had the money.

They have cool features, such as routing your mobile data (if you use
their sims on O2 network) through your broadband, which admitidly could
be done through a VPN tunnel, but I still think it is cool. Not only
that but the fact A&A does a "bring your own line" using L2TP, where I
caan use the gigabit coax provided by virgin media, and get static IPv4
and Ipv6 support as well is amazing, cause virgin media still hasn't
seemed to realise that IPv4 exhaustion exists, and people are migrating
to IPv6, and them hiding behind their huge IPv4 blocks is not moving
with the times, at least A&A realise IPv6 is the future and actually
proactively support it.

> if things like that aren't needed there are other ISPs who are also
> equally good (and better for some things) who are often a bit less dear
> (though most of those start getting expensive if you want decent size
> routed blocks, which a&a don't specifically charge for).

Actually, even then I would still go with A&A. I find their privacy
policies easy to understand, I love how open they are, and although you
cant trust them fully, I trust A&A as an transparent ISP more than any
other ISP, I can browse any website I like without having them step in
and saying "nope", also they dont block torrenting, a lot of ISPs in the
UK block torrenting.

They also have useful tools, very friendly support team. Wait on the
topic of support, how many isps do you know which the staff members will
chat to you via IRC, or GPG encrypted emails? No proprietary web
support, and I am not forced into calling them, and as an Autistic who
struggles with verbal communication, I would say being able to do
everything text based, without any phone calls makes the high price tag
more than acceptable. More companies should follow suit providing email
or open source text based support, so many companies want you to call
them and I simply can't do that.

Of course there is downsides, community fibre is shipping fibre to the
entirety of London (eventually), but I still wouldn't migrate (or if I
did I would go with L2TP) because the same level of support just isn't
there. I hate how they are limited by BT Openreach's lack of fibre
rollout, and plus its asymmetric fibre too, and CityFibre looks
promising, but they are way too small.

I am aware how expensive, I will compare two options for leased lines
below (I am aware this is off topic, but it might help someone 10 years
in the future when they laugh at how old DSL is :P):

Both options will be for 100mbps symmetric leased line to my property:

IDNET:

- Installation costs are free with the contract, they will cover them.
- £200/month

A&A:

- £2387 installation fee
- £335/month
- 1 year minimum term (I do not know IDNET's minimum term)

so comparing them both, A&A is bankrupting and is considerably more
expensive. But IRC support is just such a premium feature I would still
go with A&A.

Call me crazy, but I also spent a premium to order a custom laptop
(coming Monday O.O) from a small company because it was corebooted, I
could have picked up one a lot cheaper, but I rather pay more to support
a smaller company which follows my morals than cheap out for the bang
for the buck and support stuff I do not believe in.

Sure you can argue that most people can't throw away money on a premium,
which is understandable, but if you can afford it, why not?

I would argue A&A, apart from insanely high price, lack of their own
infrastructure, and other small issues, still makes them one of the
best, if not the best ISP in the UK. I assume most people in the OpenBSD
community is passionate about privacy, and also doing stuff yourself,
A&A are open to helping with custom routers, ask a big ISP to help you
and they will tell you to use what they shipped in the box.

I am aware Idnet and a few other ISPs are compelling alternatives,
especially that they are cheaper and do not have a quota, but as with a
lot of things, whatever you personally prefer is what you should go with :)

(Sorry for going off topic)

> beware the implicit default rule if no others match is "pass flags any
> no state" so in nearly all cases you do want a "block any" or "block log
> any" to catch those.

Yup, block all is the first rule, changes it from blacklisting rules to
whitelisting (only allowing packets you explicitly want to pass through,
and any edge cases are dropped). Just don't use "quick" with block all,
that would be tragic.

>
> it's not that - for IPv4, using just plain "inet" overrides the existing
> address; you need "alias" to add more than 1 address to an interface.
> (it's different for IPv6).

I am aware of this, but I can't remember exactly where it is stated, I
believe in one of the RFCs, but it was referenced by the IPv6
specification, paraphrased as "IPv6, unlike IPv4, can have multiple
blocks on a single interface".

I didn't know alias existed before someone recommended I use it and then
I read about it in the manpage (hostname.if(5)).

> not really from just a written description. something might come to
> mind if I see ifconfig -A, pf.conf, netstat -rnfinet, not sure though.

I already posted most of this in my original email, but it does not hurt
to include it, prepare for massive email length below this point.

Output of ifconfig -A:

PolarRouter$ ifconfig -A
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
index 3 priority 0 llprio 3
groups: lo
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
bse0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr dc:a6:32:78:eb:b1
index 1 priority 0 llprio 3
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
inet 217.169.18.57 netmask 0xfffffff8 broadcast 217.169.18.63
inet6 fe80::dea6:32ff:fe78:ebb1%bse0 prefixlen 64 scopeid 0x1
inet6 2001:8b0:57a:2385::1 prefixlen 64
enc0: flags=0<>
index 2 priority 0 llprio 3
groups: enc
status: active
bwfm0:
flags=808c43<UP,BROADCAST,RUNNING,OACTIVE,SIMPLEX,MULTICAST,AUTOCONF4>
mtu 1500
lladdr dc:a6:32:78:eb:b2
index 4 priority 4 llprio 3
groups: wlan
media: IEEE802.11 autoselect hostap (autoselect mode 11n hostap)
status: active
ieee80211: nwid PolarianWifi chan 9 bssid dc:a6:32:78:eb:b2
0dBm wpakey wpaprotos wpa2 wpaakms psk wpaciphers ccmp wpagroupcipher ccmp
inet 192.168.3.1 netmask 0xffffff00 broadcast 192.168.3.255
axen0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 7c:c2:c6:44:18:53
index 5 priority 0 llprio 3
media: Ethernet autoselect (100baseTX full-duplex)
status: active
pppoe0:
flags=248851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,AUTOCONF6TEMP,AUTOCONF6>
mtu 1492
index 6 priority 0 llprio 3
dev: axen0 state: session
sid: 0x17 PADI retries: 3 PADR retries: 0 time: 1d 15:50:15
sppp: phase network authproto chap
dns: 1.1.1.1 127.0.0.1
groups: pppoe egress
status: active
inet6 fe80::dea6:32ff:fe78:ebb1%pppoe0 --> prefixlen 64
scopeid 0x6
inet 81.187.86.85 --> 81.187.81.187 netmask 0xffffffff
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33136
index 7 priority 0 llprio 3
groups: pflog

(Note: bwfm0 is unused, it does not work due to a firmware issue, I
believe I know how to fix it but it means reinstalling OpenBSD)

(Note: axen0 is the WAN interface, which the pppoe0 interface sits upon.
bse0 is the LAN interface)

Contents of pf.conf:


lan = "bse0"
wan = "pppoe0"

###### Servers and Containers ######

# Stipped, contained names of people who would rather not have their
name leaked on a public mailing list (sorry)

###### IPv4 allocations ######

nataddr = "81.187.86.85"
staticv4b1 = "217.169.18.56/29"

###### IPv6 allocations ######

alloc1 = "2001:8b0:57a:2385::/64"

# block log

block log

# Allow all ICMP traffic

pass quick proto icmp6
pass in quick on $wan proto icmp
pass out quick on $lan proto icmp

###### IPv6 ######

# Deny all incoming traffic to router

block in quick on $wan from any to 2001:8b0:57a:2385::1/128

# Allow traffic on port 5000 (wireguard) to router

pass in quick on $wan proto udp from any to $nataddr port {5000}
pass out quick on $wan from (wg0:network) to any nat-to $nataddr

# Allow network discovery

pass in quick on $lan from fe80::/10 to fe80::/10
pass out quick on $lan from fe80::/10 to fe80::/10

# Pass out all inet6 traffic

pass in on $lan inet6 from $alloc1 to any
pass in on $wan inet6 from any to $alloc1
pass out on $wan inet6 from $alloc1 to any

# Pass traffic to LAN

pass out on $lan inet6 from any to $alloc1

###### IPv4 ######

# Static v4 rules

pass in quick on $wan inet from $staticv4b1 to any keep state
pass in quick on $lan inet from $staticv4b1 to any keep state
pass out quick on $lan inet from any to $staticv4b1 keep state
pass out quick on $wan inet from $staticv4b1 to any keep state

pass in on $lan inet from $lan:network to any keep state
pass out on $wan inet from $lan:network to any nat-to $nataddr keep state

# Allow firewall to communicate with internet

pass out on $wan inet from $wan:network to any keep state

###### Port Forwards ######

# Stripped, contained notes and names which again, people would not
appreciated posted on a public mailing list

# Pass all port forwards to lan

pass out on $lan
pass out on $lan from 192.168.2.1/24 received-on $lan nat-to 192.168.2.1

(Note: Some irrelevant sections stripped due to the me being lazy and
using real names as comments to identify specific addresses and port
forward rules)

(Note: The initial IPv4 NAT rules was from the manpages, I have not
touched this since then, the rest of the rules were written by me, so if
you have any optimisations let me know, I am always happy to know how to
improve. I think the pf.conf reflects my experience, which is almost
none when it comes to OpenBSD :/)

(Note: I have clearly commented everything, mainly to stop myself from
getting confused, but also so others who read it can actually understand
it, if any comment doesn't make sense, let me know :))

Output of netstat -rnfinet:

PolarRouter$ netstat -rnfinet
Routing tables

Internet:
Destination Gateway Flags Refs Use Mtu Prio
Iface
default 81.187.81.187 UGS 989 15800769 - 8
pppoe0
224/4 127.0.0.1 URS 0 0 32768 8 lo0
81.187.81.187 81.187.86.85 UHh 1 1 - 8
pppoe0
81.187.86.85 81.187.86.85 UHl 0 37744 - 1
pppoe0
127/8 127.0.0.1 UGRS 0 0 32768 8 lo0
127.0.0.1 127.0.0.1 UHhl 1 3 32768 1 lo0
192.168.2/24 192.168.2.1 UCn 19 26 - 4 bse0
192.168.2.1 dc:a6:32:78:eb:b1 UHLl 0 333089 - 1 bse0
192.168.2.2 e2:1b:81:ac:c7:a1 UHLc 0 2271865 - 3 bse0
192.168.2.3 a4:34:d9:78:c5:94 UHLc 0 27924 - 3 bse0
192.168.2.11 00:16:3e:38:e1:e0 UHLc 0 110587 - 3 bse0
192.168.2.15 00:16:3e:e7:6e:d8 UHLc 0 50591 - 3 bse0
192.168.2.16 link#1 UHRLc 0 33 - 3 bse0
192.168.2.17 52:54:00:d3:ec:64 UHLc 0 143933 - 3 bse0
192.168.2.18 00:16:3e:fd:34:cc UHLc 0 73532 - 3 bse0
192.168.2.19 00:16:3e:22:c3:58 UHLc 0 72711 - 3 bse0
192.168.2.20 link#1 UHLc 0 34 - 3 bse0
192.168.2.25 cc:2d:21:ba:fe:d0 UHLc 0 10777 - 3 bse0
192.168.2.26 e8:65:d4:64:59:80 UHLc 0 178645 - 3 bse0
192.168.2.78 74:56:3c:67:bf:35 UHLc 1 1966774 - 3 bse0
192.168.2.80 00:16:3e:62:88:54 UHLc 0 7977 - 3 bse0
192.168.2.81 bc:60:a7:99:c6:05 UHLc 0 6016 - 3 bse0
192.168.2.151 aa:01:c3:46:e7:88 UHLc 1 560487 - 3 bse0
192.168.2.155 cc:2d:21:ba:fe:d0 UHLc 0 727434 - 3 bse0
192.168.2.157 ea:ff:49:20:be:3e UHLc 0 61276 - 3 bse0
192.168.2.158 12:92:59:5a:30:21 UHLc 0 991 - 3 bse0
192.168.2.161 c8:3a:35:a4:08:ef UHLc 0 253689 - 3 bse0
192.168.2.255 192.168.2.1 UHb 0 234 - 1 bse0
192.168.3/24 192.168.3.1 UCn 0 0 - 8
bwfm0
192.168.3.1 dc:a6:32:78:eb:b2 UHLl 0 0 - 1
bwfm0
192.168.3.255 192.168.3.1 UHb 0 0 - 1
bwfm0
217.169.18.56/29 217.169.18.57 UCn 1 0 - 4 bse0
217.169.18.57 dc:a6:32:78:eb:b1 UHLl 0 137299 - 1 bse0
217.169.18.58 00:16:3e:bf:b5:92 UHLc 0 52502 - 3 bse0
217.169.18.63 217.169.18.57 UHb 0 90 - 1 bse0

I am sure the problem is an obvious mistake somewhere due to lack of
experience, I am sure its going to be a big facepalm when the issue is
found :P

Anyways, thank you again for all the help.

Have a good evening,
--
Polarian
GPG signature: 0770E5312238C760
Website: https://polarian.dev
JID/XMPP: polarian@polarian.dev

No comments:

Post a Comment