Well if you are using docker-compose you could probably get rid of the nginx container and only deploy the other four: lemmy, lemmy-ui, pictrs, postgres. You would then use the nginx.conf stuff you have in place for the docker container of nginx to proxy to lemmy-ui and lemmy on ports 1234 and 8536. Or if you plan to keep using the docker container for nginx then you can change the listening port in the nginx.conf of the container:
listen 80;
to something different like
listen 1080;
Also in the docker-compose.yml you would update the nginx ports to 1080:1080
.
Hope this answers your question.
You are still giving them traffic, just not directly.