Tuesday, November 30, 2021

Re: Running redmine on OpenBSD

W dniu 30.11.2021 o 16:07, Radek pisze:
> On Tue, 30 Nov 2021 10:04:30 +0100
> Łukasz Moskała <lm@lukaszmoskala.pl> wrote:
>
>>
>>
>> Dnia 30 listopada 2021 09:45:15 CET, Radek <rdk@int.pl> napisał/a:
>>> On Mon, 29 Nov 2021 11:19:28 +0100
>>> Łukasz Moskała <lm@lukaszmoskala.pl> wrote:
>>>
>>>> W dniu 28.11.2021 o 18:07, Radek pisze:
>>>>> Hello,
>>>>> following the official guide [1] and few others webites I finally installed my first Ruby on Rails/Puma web app... and it passed the local test by curl (bundle exec rails server webrick -e production) - relayd wasn't configured yet.
>>>>>
>>>>> Then, I ran my app with puma server. I can't figure out how to make it work with FQDN and LetsEncrypt cert.
>>>>> My configs seems to be fine. It's 7.0/amd64. I've read [2], [3].
>>>>>
>>>>> I started with simple httpd configuration to get certs with acme-clinet and then https://redmine.MY.DOMAIN.COM showed my testing index.html properly.
>>>>> Now /etc/httpd.conf has changed but I assume my certs are still OK.
>>>>>
>>>>> Remote firefox is giving me a "Redirect Loop" error when trying to access https://redmine.MY.DOMAIN.COM
>>>>>
>>>>> Could someone please shed some light on this puzzle?
>>>>>
>>>>> 1. https://www.redmine.org/projects/redmine/wiki/RedmineInstall
>>>>> 2. https://github.com/basicfeatures/openbsd-rails
>>>>> 3. https://gist.github.com/anon987654321/4532cf8d6c59c1f43ec8973faa031103
>>>>>
>>>>> $ openssl s_client -connect redmine.MY.DOMAIN.COM:443
>>>>> CONNECTED(00000003)
>>>>> depth=0 CN = redmine.MY.DOMAIN.COM
>>>>> verify error:num=20:unable to get local issuer certificate
>>>>> verify return:1
>>>>> depth=0 CN = redmine.MY.DOMAIN.COM
>>>>> verify error:num=21:unable to verify the first certificate
>>>>> verify return:1
>>>>> write W BLOCK
>>>>> ---
>>>>> Certificate chain
>>>>> 0 s:/CN=redmine.MY.DOMAIN.COM
>>>>> i:/C=US/O=Let's Encrypt/CN=R3
>>>>> ---
>>>>> Server certificate
>>>>> -----BEGIN CERTIFICATE-----
>>>>> [...]
>>>>> -----END CERTIFICATE-----
>>>>> subject=/CN=redmine.MY.DOMAIN.COM
>>>>> issuer=/C=US/O=Let's Encrypt/CN=R3
>>>>> ---
>>>>> No client certificate CA names sent
>>>>> Server Temp Key: ECDH, X25519, 253 bits
>>>>> ---
>>>>> SSL handshake has read 2403 bytes and written 367 bytes
>>>>> ---
>>>>> New, TLSv1/SSLv3, Cipher is AEAD-AES256-GCM-SHA384
>>>>> Server public key is 4096 bit
>>>>> Secure Renegotiation IS NOT supported
>>>>> Compression: NONE
>>>>> Expansion: NONE
>>>>> No ALPN negotiated
>>>>> SSL-Session:
>>>>> Protocol : TLSv1.3
>>>>> Cipher : AEAD-AES256-GCM-SHA384
>>>>> Session-ID:
>>>>> Session-ID-ctx:
>>>>> Master-Key:
>>>>> Start Time: 1638116582
>>>>> Timeout : 7200 (sec)
>>>>> Verify return code: 21 (unable to verify the first certificate)
>>>>> ---
>>>>>
>>>>>
>>>>> [redminepk@@redmine70~/redminepk:]bundle exec pumactl27 --config-file config/puma.rb start
>>>>> Puma starting in single mode...
>>>>> * Puma version: 5.5.2 (ruby 2.7.4-p191) ("Zawgyi")
>>>>> * Min threads: 0
>>>>> * Max threads: 5
>>>>> * Environment: production
>>>>> * PID: 85983
>>>>> * Listening on ssl://127.0.0.1:3000?cert=/etc/ssl/redmine.MY.DOMAIN.COM.crt&key=/etc/ssl/private/redmine.MY.DOMAIN.COM.key&verify_mode=none
>>>>> * Listening on http://127.0.0.1:3001
>>>>> Use Ctrl-C to stop
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> # /home/redminepk/redminepk/config/puma.rb
>>>>> #!/usr/bin/env puma
>>>>> app = "redminepk"
>>>>> ssl_bind "127.0.0.1", "3000", {
>>>>> key: "/etc/ssl/private/redmine.MY.DOMAIN.COM.key",
>>>>> cert: "/etc/ssl/redmine.MY.DOMAIN.COM.crt"
>>>>> }
>>>>> bind "tcp://127.0.0.1:3001"
>>>>> pidfile "/home/#{app}/#{app}/tmp/puma.pid"
>>>>> state_path "/home/#{app}/#{app}/tmp/puma.state"
>>>>> stdout_redirect "/home/#{app}/#{app}/log/puma_access.log", "/home/#{app}/#{app}/log/puma_errors.log"
>>>>> environment "production"
>>>>>
>>>>>
>>>>> # /home/redminepk/redminepk/config/environments/production.rb
>>>>> Rails.application.configure do
>>>>> config.cache_classes = true
>>>>> config.eager_load = true
>>>>> config.consider_all_requests_local = false
>>>>> config.action_controller.perform_caching = true
>>>>> config.action_mailer.raise_delivery_errors = false
>>>>> config.action_mailer.logger = nil
>>>>> config.active_support.deprecation = :log
>>>>> config.force_ssl = true
>>>>> end
>>>>>
>>>>>
>>>>>
>>>>> # /etc/httpd.conf
>>>>> ext_if="vmx0"
>>>>> types { include "/usr/share/misc/mime.types" }
>>>>> server "redmine.MY.DOMAIN.COM" {
>>>>> listen on $ext_if port 80
>>>>> location "/.well-known/acme-challenge/*" {
>>>>> root "/acme"
>>>>> request strip 2
>>>>> }
>>>>> location "*" {
>>>>> block return 302 "https://$HTTP_HOST$REQUEST_URI"
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>>> # /etc/relayd.conf
>>>>> egress="A.B.C.D"
>>>>> table <redminepk> { 127.0.0.1 }
>>>>> redminepk_port="3001"
>>>>> table <httpd> { 127.0.0.1 }
>>>>> httpd_port="80"
>>>>> http protocol "http" {
>>>>> match request header set "Connection" value "close"
>>>>> match response header remove "Server"
>>>>> }
>>>>> http protocol "https" {
>>>>> pass request header "Host" value "redmine.MY.DOMAIN.COM" forward to <redminepk>
>>>>> tls keypair "redmine.MY.DOMAIN.COM"
>>>>> # Preserve address headers
>>>>> match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
>>>>> match request header append "X-Forwarded-Port" value "$REMOTE_PORT"
>>>>> match request header append "X-Forwaded-By" value "$SERVER_ADDR:$SERVER_PORT"
>>>>> match request header set "Connection" value "close"
>>>>> match response header remove "Server"
>>>>> }
>>>>> relay "http" {
>>>>> listen on $egress port http
>>>>> protocol "http"
>>>>> forward to <httpd> port $httpd_port
>>>>> }
>>>>> relay "https" {
>>>>> listen on $egress port https tls
>>>>> protocol "https"
>>>>> forward to <httpd> port $httpd_port
>>>>> forward to <redminepk> port $redminepk_port
>>>>> }
>>>>>
>>>>> $ grep relayd /etc/pf.conf
>>>>> # Allow relayd(8) redirects
>>>>> anchor "relayd/*"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, 12 Nov 2021 20:35:45 +0100
>>>>> Radek <rdk@int.pl> wrote:
>>>>>
>>>>>> Hello Werner,
>>>>>> thank you for your installation details. I'll give it a try in a few days.
>>>>>>
>>>>>> On Thu, 11 Nov 2021 23:57:02 +0800
>>>>>> Werner Boninsegna <werner@dewrico.com> wrote:
>>>>>>
>>>>>>> Hello Radek,
>>>>>>>
>>>>>>> I am running Redmine on OpenBSD 6.8 and I just followed the installation
>>>>>>> instructions posted on the Redmine page which are quite complete:
>>>>>>>
>>>>>>> https://www.redmine.org/projects/redmine/wiki/Installation_Guide
>>>>>>>
>>>>>>> I installed Postgres and Ruby+Dependencies from the OpenBSD packages.
>>>>>>>
>>>>>>> Werner
>>>>>>>
>>>>>>> On 11/10/21 00:56, Radek wrote:
>>>>>>>
>>>>>>> Hi @misc,
>>>>>>> Does anyone successfully run redmine[1] on OpenBSD?
>>>>>>> I'd like to install redmine on 7.0/amd64 with httpd and postgresql. I've never done it before so any advices and hints would be appreciated.
>>>>>>> There isn't much up to date info in google about it[2][3].
>>>>>>>
>>>>>>> 1. https://www.redmine.org/ 2. https://www.redmine.org/boards/2/topics/496 3. https://web.archive.org/web/20160406041905/http://www.iwebdev.it/blog/?p=229
>>>>>>> Thank you!
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Radek
>>>>>>
>>>>>
>>>>>
>>>> Hi,
>>>> I have never used relayd (only haproxy), so some things that I say may
>>>> be wrong. Sorry about that in advance.
>>>>
>>>> First of all, what's the point of passing http via relayd?
>>>>
>>>> Now, going back to your issue,why do you have
>>>> >forward to <httpd> port $httpd_port
>>>> in relay "https" ?
>>>>
>>>> It looks like relayd may put those requests to httpd, which will
>>>> redirect them to https.
>>>>
>>>> So, I'd start looking at httpd and puma logs, to check to which daemon
>>>> the requests are actually being sent to.
>>>>
>>>> And, if i'm wrong, and requests are going to puma (or whatever is
>>>> listening on 3001), then most likely puma is trying to redirect users to
>>>> https. Possibly because of this line:
>>>> >config.force_ssl = true
>>>>
>>>> Also, in this case you may try setting header x-forwarded-proto to
>>>> https, but I don't know if puma will actually use it.
>>>>
>>>>
>>>> Let us know if that worked.
>>>>
>>>> Kind regards,
>>>> --
>>>> Łukasz Moskała
>>> Hello Łukasz,
>>>
>>>> First of all, what's the point of passing http via relayd?
>>> I used to think that all http and https traffic have to be passed via relayd if relayd is started.
>>> I have never used relayd before.
>>>
>>>> Now, going back to your issue,why do you have
>>>> >forward to <httpd> port $httpd_port
>>>> in relay "https" ?
>>> I think it was pasted there by mistake. I removed this line.
>>>
>>>> So, I'd start looking at httpd and puma logs, to check to which daemon
>>>> the requests are actually being sent to.
>>> Unfortunately, there is nothing in httpd and puma logs.
>>> The only logs I have are from relayd (-dvv):
>>> relay https, tls session 1 established (1 active)
>>> relay_tls_transaction: session 2: scheduling on EV_READ
>>> relay https, tls session 2 established (1 active)
>>> relay_tls_transaction: session 3: scheduling on EV_READ
>>> relay https, tls session 3 established (1 active)
>>> [...]
>>>
>>>> And, if i'm wrong, and requests are going to puma (or whatever is
>>>> listening on 3001), then most likely puma is trying to redirect users to
>>>> https. Possibly because of this line:
>>>> >config.force_ssl = true
>>> I suppose it should work this way but I don't know what actually is going on there between puma and relayd.
>>>
>>>> Also, in this case you may try setting header x-forwarded-proto to
>>>> https, but I don't know if puma will actually use it.
>>> If the below syntax is correct this option doesn't make any change.
>>> match request header set "X-Forwaded-Proto" value "$SERVER_PORT"
>>>
>>>
>>
>> Now, if I think about it again, you can run tcpdump on lo0, and hopefully see what is going on.
>>
>> Also, I feel like in your case, relayd is unnecessary here, as you can run httpd on port 80, for redirecting to https and acme, and then run puma on port 3000, and use pf to NAT traffic coming on vmx0 to EXT_IP port 443 to 127.0.0.1 port 3000.
>>
>> It would look more or less like this (typing on phone from memory, may not be exactly correct):
>> pass in quick on vmx0 inet proto tcp from any to $EXT_IP rdr-to 127.0.0.1:3000
>>
>> This way http requests will still be served by httpd, and https requests will go straight to puma.
>>
>> I don't think you need to enable ip forwarding for that, but you can try it if it doesn't work.
>>
>> also, you can try going to https://EXT_IP:3000 and http://EXT_IP:3001, to see what will happen. curl -v will probably show more than browsers.
>>
>> --
>> Łukasz Moskała
>
> I care about configuring it with relayd. It's a good opportunity to learn this field.
> If I can't handle it I will try redirecting 443 to puma/localhost with httpd and PF.
>
> Going back to case with enabled relayd..
> curl -v https://redmine.MY.DOMAIN.COM shows:
> * Expire in 0 ms for 6 (transfer 0x5654a296bfb0)
> * Expire in 1 ms for 1 (transfer 0x5654a296bfb0)
> [...]
> * Expire in 15 ms for 1 (transfer 0x5654a296bfb0)
> * Expire in 50 ms for 1 (transfer 0x5654a296bfb0)
> * Trying A.B.C.D...
> * TCP_NODELAY set
> * Expire in 200 ms for 4 (transfer 0x5654a296bfb0)
> * Connected to redmine.MY.DOMAIN.COM (A.B.C.D) port 443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * successfully set certificate verify locations:
> * CAfile: none
> CApath: /etc/ssl/certs
> * TLSv1.3 (OUT), TLS handshake, Client hello (1):
> * TLSv1.3 (IN), TLS handshake, Server hello (2):
> * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
> * TLSv1.3 (IN), TLS handshake, Certificate (11):
> * TLSv1.3 (OUT), TLS alert, unknown CA (560):
> * SSL certificate problem: unable to get local issuer certificate
> * Closing connection 0
> curl: (60) SSL certificate problem: unable to get local issuer certificate
> More details here: https://curl.haxx.se/docs/sslcerts.html
>
> tcpdump -i lo0 shows nothing
>
>
> relayd -dvv shows:
> relay_tls_transaction: session 5: scheduling on EV_READ
> relay_close: sessions inflight decremented, now 0
>
>

Curl did connect to relayd, and didn't send any request, because it
didn't accept certificate presented by relayd, so relayd didn't try to
pass this any further.

It looks like relayd didn't send all intermidiate certificates, thus
curl was not able to find complete trust chain, resulting in this error.
You can deal with it later, for now you can add option -k or --insecure
to curl, to ignore certificate verification.

Then you could look again in tcpdump, logs, and curl output.

Again, I think you are really making your life difficult trying to use
relayd. Relayd is supposed to be used as a load-balancer running on a
router, to split traffic between multiple servers, optionally
terminating TLS connection, but in your case there's only one server, so
no need to load-balance, and puma can listen on TLS on it's own, so no
need to terminate TLS connection.
Also it will add you a little bit of overhead, since relayd in layer7
has to accept the connection, and then relayd opens another connection
to puma.

--
Łukasz Moskała

No comments:

Post a Comment