feat: extract nginx into separate container - #430
Conversation
|
Great work on this PR, @pascaliske! The separation of nginx into its own container is a much-needed architectural improvement and the security hardening (non-root, read-only fs, dropped capabilities) is excellent. The move to nginxinc/nginx-unprivileged with envsubst-based templates is much cleaner than the old sed-based approach.
|
|
Thanks for the feedback - I will have a look into and address those points. |
|
Sounds good, thanks! Let me know if you want me to re-review once you've pushed updates |
|
@mortezamirkar I addressed your points:
The removal of ports
Forgot to switch back after debugging, I set this to error again - easy fix 😄.
The FastCGI params are actually included via
Forgot to add this back as well. Actually I just
I re-added the logic for supporting IPv6 and for disabling it via
I moved the FPM status page route alongside the nginx stub status to a separate port which is only reachable from within the nginx container (
Yeah makes sense - I rewrote the health check to check against the status page. This prevents unnecessary restarts of nginx if the actual FPM container fails - static assets are still served.
Done. 🙂 Could you please assist in re-testing the PR? Thank you! |
c71739d to
fca2ae8
Compare
|
Just a quick comment (especially related to the TLS part): we still need this to be a zero to hero container, so TLS part needs to be handled in one way or another. Also we need to make this backwards compatible in some way. Can't we add another container taking care of that part? |
I already thought about making it based on the presence of cert & key files. Basically if those two files are present it also enables serving via 8443 with those cert in-place. In the What do you think about this idea @ostefano? |
| tags = flatten(["${NAMESPACE}/misp-nginx:latest", "${NAMESPACE}/misp-nginx:${COMMIT_HASH}", NGINX_TAG != "" ? ["${NAMESPACE}/misp-nginx:${NGINX_TAG}"] : []]) | ||
| args = { | ||
| "NGINX_TAG": "${NGINX_TAG}", | ||
| "NGINX_COMMIT": "${NGINX_COMMIT}", |
There was a problem hiding this comment.
I don't understand what is the role of NGINX_COMMIT.
Should the only variable here be version of the base image?
There was a problem hiding this comment.
Yes, it's just for referencing the source code in the Dockerfile. Should I just use CORE_TAG & CORE_COMMIT here as well?
There was a problem hiding this comment.
Gave it a proper look and I am keen to say we just need CORE_TAG and CORE_COMMIT after all.
@mortezamirkar what's your take?
There was a problem hiding this comment.
I decided to introduce nginx specific variables but set it to the values of their core counterparts in .env just to make this explicit that it uses the same versioning. But I would also be fine with just using CORE_TAG & CORE_COMMIT directly inside the nginx stuff - I think it's up to you to decide as the maintainers. 🙂
I have re-implemented the TLS feature. It is based on the presence of the certificate and key:
EDIT: The redirect is also re-added but due to the split it requires the |
41a2508 to
95ee4d1
Compare
There was a problem hiding this comment.
We need to decode how the NGINX tag / commit will play with the existing CI/CD.
Normally they map a specific tag / commit in the project repository, but that of course would not apply to nginx. Should we use a separate versioning?
A concern we need to explore is also how do we support existing setups. Would they get automatically migrated? I.e., is the current solution backward compatible?
|
Also this have an impact to everything experiment and/or related to kubernetes. We should update those manifests. |
Seems like this will be the way. I will prepare that.
I will have a look into this and prepare some documentation for that.
The Docker updates actually are just in preparation for something I've been working on lately which improves the deployment and configuration of any count of MISP instances inside Kubernetes. It's native Kubernetes tooling and integrates perfectly with the ecosystem. But additionally I will update the plain manifests when the actual change is finalized. 👍🏻 |
|
Great iteration on this PR, @pascaliske! The TLS re-implementation based on cert/key presence is a clean approach, and the security hardening (read-only FS, cap_drop: ALL, no-new-privileges, nginx-unprivileged image) is well done. The envsubst-based templating is significantly cleaner than the old sed-based config mutation.
|
|
Thanks for your additional and thorough review! I updated my code accordingly:
Done. I also ensured that the target directory exists.
I will have a look into this later this evening and keep you updated.
That's actually true for all includes except
I switched the user back to nginx and ensured the directory ownership for
Yeah that was a pain point. Due to the separation the webroot must be served from nginx directly and therefore the
Good catch. The log here is unnecessary and I removed it.
Should I remove that as well? I thought it would be nice to make sure the core image is ready because nginx without the core is basically useless... 😁
Yeah I updated the branch based on the latest master and squashed everything into one commit. I will do a second commit which highlights the breaking changes from your 2nd point so that it's easily visible inside the history additionally. |
19b5130 to
dfef4af
Compare
|
@mortezamirkar I added a block to the PR description which highlights |
|
@pascaliske this is a breaking change on several levels, so it might take a while. I plan to spend on it some quality cycles soon though. |
|
Great iteration, @pascaliske . The security posture (non-root, read-only FS, I've re-reviewed the latest changes: All previous items addressed -- TLS, error_log, FastCGI params, submodule Two minor items I noticed:
Remaining blockers (from @ostefano): Beyond that, this is ready from my side. Happy to re-review once the above |
dfef4af to
8a04e08
Compare
Done.
I removed the variable
I re-used the variables Regarding the backward compatibility documentation I have a question: Where and how should I document that? I have added a block to the PR description about the breaking changes in the |
|
@pascaliske I've re-reviewed the updated PR and all feedback points have been properly addressed. The changes look solid:
LGTM - this is ready to merge. Great work on the architecture separation! |
|
I have started looking into it, and a [Of course I can look at the changed files, but I am replicating what a user would do] |
Yes, you're right. The HTTPS port now is commented out by default in the services:
misp-nginx:
ports:
- "${NGINX_HTTP_PORT:-80}:8080"
# uncomment and make sure to have the cert & key in-place to enable https support
# - "${NGINX_HTTPS_PORT:-443}:8443"You have to explicitly provide a certificate and uncomment the line to enable HTTPS, since the container no longer has an automatically generated, self-signed certificate. Is this a blocker for you? Probably this should be mentioned somewhere else as well? |
|
Ideally: Level 1 (would say mandatory): no-op if user is using old env file and new docker compose. |
8a04e08 to
a80efc4
Compare
|
I changed the volume paths for the cert files in the nginx container. Now the ssl files from the previous core container should be used inside nginx automatically therefore I could also enable the HTTPS port by default. This should be Level 1 as you wrote. |
a80efc4 to
fc82f12
Compare
|
Lots of our users use https://podman.io/ . Did anybody try to give it a spin and see if the new changes play nice with it? |
|
@pascaliske @ostefano I tried running it following these steps: fc82f12 However, as shown below, the It seems like Looking at this line, it seems to assume that My apologies if I missed a step in the setup process. I’d appreciate it if you could take a look.
|
fc82f12 to
496ac00
Compare
|
One question from my side: at the moment |
Signed-off-by: Pascal Iske <info@pascaliske.dev>
Signed-off-by: Pascal Iske <info@pascaliske.dev>
496ac00 to
2d27ddf
Compare
@fukusuket Thanks for testing! Yes you're right it seems the health check got accidentally broken during the development and I missed it. I switched it to a TCP based check which has no extra dependencies than bash (which is available inside |
Actually the same command should give you a working instance too after the change. The only things that need to be changed are the minimal env variables ( Generating a self-signed certificate in a production image (as it was before) just for testing out an image is not worth it so I thought it would be an acceptable trade-off. But it would be possible to just provide an example command in the |
|
@pascaliske Thank you! However, the log still shows HTTP 400 errors, and I cannot access http://localhost:8080 yet. |
Got it, I understand your approach now. Yes, providing that in the README sounds like a good plan! |
No. You published port 80 as you can see in the So you should actually be able to access the instance at http://localhost:80. 🙂 |
|
Oh, my apologies! Added this to my |
|
@pascaliske |
|
Might be because the image |
Yes, as @ostefano said the image isn't pushed. You need to run |
|
I see! I've confirmed that |

Hi MISP-team,
Thank you for your work in this repository.
This PR moves nginx from the
misp-corecontainer to a new separatemisp-nginxcontainer.The intention behind it is:
→ The nginx process now fully runs with a non-root user in a read-only root filesystem
→ Linux capabilities are dropped completely
→ Privileges are limited to a minimum
→ Separation of concerns, leaving grouping / orchestration to infra-tooling (Docker Compose / Kubernetes / etc.)
→ Easier horizontal scaling due to separation
Please note: the high number of line changes in the misp-core Dockerfile are due to some re-ordering and whitespace / indentation changes I made, I hope this is fine for you.
Closes #340.
Attention: Following changes to
.envvariables are introduced: