Tikejhya: Ashish Nepal

Knowledgebase

Category: nginx

Nginx Multiple satisfy Any [auth_request]

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

We had usecase where we needed to allow ip based or authentication via username and password.

This was all working with http://www.ashishnepal.com/nginx-restricting-access/ [statisfy Any with auth_basic and allow deny]

However there was certain access required to provide using UA (user_agent) which was not possible to do from IP or even to use username and password based authentication.

This could have been achieved in multiple ways: 1) using nginx @forwarder and sending 403 2) mapping call with user_agent and geo 3) using different backend with use of HAproxy or Varnish or even using any gateway and spoofing x-real-ip or x-forwarded-for and manipulating accordingly

But most elegant way would be nginx’s module auth_request.
Here Nginx comes very useful with auth_request.

location / {
satisfy any;

allow 192.168.x.x/24;
deny all;

auth_basic "Require Auth";
auth_basic_user_file conf.d/htpasswd;
auth_request /authentication;
}

location = /authentication {
if ($http_user_agent ~ canHazUA){
return 200;
}
return 403;
}

nginx Restricting Access

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

Restricting Access using allow deny with satisfy Any.

Access can be allowed or denied based on the IP address of a client or by using HTTP basic authentication.

To allow or deny access from a certain set of addresses, or all addresses, use the allow and deny directives:

location / {
allow 192.168.1.1/24;
allow 127.0.0.1;
deny 192.168.1.2;
deny all;
}

Restricting access with allow deny can be complemented with use of http://nginx.org/en/docs/http/ngx_http_auth_basic_module.html

Your ads will be inserted here by

Easy Plugin for AdSense.

Please go to the plugin admin page to
Paste your ad code OR
Suppress this ad slot.

which allows to use either IP based restriction or User password based authentication. This is one of the key feature in Apache and majority of we want to have simillar use case senario:

location / {
auth_basic "closed site";
auth_basic_user_file conf/htpasswd;
}

So combining this two may look like this:

location / {
satisfy any;

allow 192.168.1.1/24;
allow 127.0.0.1;
deny 192.168.1.2;
deny all;

auth_basic "closed site";
auth_basic_user_file conf/htpasswd;
}

nginx basic_auth

The ngx_http_auth_basic_module module allows limiting access to resources by validating the user name and password using the “HTTP Basic Authentication” protocol.

location / {
auth_basic “closed site”;
auth_basic_user_file conf/htpasswd;
}

Nginx cache cleanup

KEY: httpGETashishnepal.com/nginx/

echo -n "httpGETashishnepal.com/nginx/" | md5sum | awk '{print "/etc/nginx/cache/"substr($1,length($1),1)"/"substr($1,length($1)-2,2)"/"$1}'

Above solution would only work if your setup of nginx cache_key is on levels=1:2

Once you get the hashed filename it can be rm -rf’ed.

Starting nginx: nginx: [warn] could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size

#Starting nginx: nginx: [warn] could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size

Background
During the start and each re-configuration nginx selects the minimum possible sizes of hash tables such that the bucket size that stores keys with identical hash values does not exceed the configured parameter (hash bucket size). The size of a table is expressed in buckets. The adjustment is continued until the table size exceeds the hash max size parameter. Most hashes have the corresponding directives that allow changing these parameters, for example, for the server names hash they are server_names_hash_max_size and server_names_hash_bucket_size.

Solution: Generall you will need to increase the recommended sizes but if you are using duplicate proxy_set_header this hash size can easily get full.

Cause: 1) too many domains to map into hash 2) too low server_name_hash size 3) too many proxy_header values to set.

Check for duplicates of proxy_set_header.

Nginx Mobile Tablet and Desktop User agent detection

How to detect Mobile tablet and desktop useragent.

# Add MAP for Default As desktop
# First match regex = mobile
# Second match regex = Tablet

map $http_user_agent $mobile_device {
    default Desktop;
  
	~*iPad|iPad.*Mobile|^.*Android.*Nexus(((?:(?!Mobile))|(?:(\s(7|10).+))).)*$|SAMSUNG.*Tablet|Galaxy.*Tab|SC-01C|GT-P1000|GT-P1010|GT-P6210|GT-P6800|GT-P6810|GT-P7100|GT-P7300|GT-P7310|GT-P7500|GT-P7510|SCH-I800|SCH-I815|SCH-I905|SGH-I957|SGH-I987|SGH-T849|SGH-T859|SGH-T869|SPH-P100|GT-P3100|GT-P3110|GT-P5100|GT-P5110|GT-P6200|GT-P7320|GT-P7511|GT-N8000|GT-P8510|SGH-I497|SPH-P500|SGH-T779|SCH-I705|SCH-I915|GT-N8013|GT-P3113|GT-P5113|GT-P8110|GT-N8010|GT-N8005|GT-N8020|GT-P1013|GT-P6201|GT-P6810|GT-P7501|Kindle|Silk.*Accelerated|xoom|sholest|MZ615|MZ605|MZ505|MZ601|MZ602|MZ603|MZ604|MZ606|MZ607|MZ608|MZ609|MZ615|MZ616|MZ617|Android.*\b(A100|A101|A110|A200|A210|A211|A500|A501|A510|A511|A700|A701|W500|W500P|W501|W501P|W510|W511|W700|G100|G100W|B1-A71)\b|Android.*(AT100|AT105|AT200|AT205|AT270|AT275|AT300|AT305|AT1S5|AT500|AT570|AT700|AT830)|Sony\ Tablet|Sony\ Tablet\ S|SGPT12|SGPT121|SGPT122|SGPT123|SGPT111|SGPT112|SGPT113|SGPT211|SGPT213|EBRD1101|EBRD1102|EBRD1201|MID1042|MID1045|MID1125|MID1126|MID7012|MID7014|MID7034|MID7035|MID7036|MID7042|MID7048|MID7127|MID8042|MID8048|MID8127|MID9042|MID9740|MID9742|MID7022|MID7010|MediaPad|IDEOS\ S7|S7-201c|S7-202u|S7-101|S7-103|S7-104|S7-105|S7-106|S7-201|S7-Slim|IQ310|Fly\ Vision|Android.*(K8GT|U9GT|U10GT|U16GT|U17GT|U18GT|U19GT|U20GT|U23GT|U30GT)|CUBE\ U8GT|Android.*(\bMID\b|MID-560|MTV-T1200|MTV-PND531|MTV-P1101|MTV-PND530)|\bL-06C|LG-V900|LG-V909\|Android.*(TAB210|TAB211|TAB224|TAB250|TAB260|TAB264|TAB310|TAB360|TAB364|TAB410|TAB411|TAB420|TAB424|TAB450|TAB460|TAB461|TAB464|TAB465|TAB467|TAB468)|Android.*\bOYO\b|LIFE.*(P9212|P9514|P9516|S9512)|LIFETAB|AN10G2|AN7bG3|AN7fG3|AN8G3|AN8cG3|AN7G3|AN9G3|AN7dG3|AN7dG3ST|AN7dG3ChildPad|AN10bG3|AN10bG3DT|Android.*ARCHOS|101G9|80G9|NOVO7|Novo7Aurora|Novo7Basic|NOVO7PALADIN|Transformer|TF101\|PlayBook|RIM\ Tablet|HTC\ Flyer|HTC\ Jetstream|HTC-P715a|HTC\ EVO\ View\ 4G|PG41200|Android.*Nook|NookColor|nook\ browser|BNTV250A|LogicPD\ Zoom2|Android.*(RK2818|RK2808A|RK2918|RK3066)|RK2738|RK2808A|bq.*(Elcano|Curie|Edison|Maxwell|Kepler|Pascal|Tesla|Hypatia|Platon|Newton|Livingstone|Cervantes|Avant)|Android.*\b97D\b|Tablet(?!.*PC)|ViewPad7|MID7015|BNTV250A|LogicPD\ Zoom2|\bA7EB\b|CatNova8|A1_07|CT704|CT1002|\bM721\b|hp-tablet|Playstation|TB07STA|TB10STA|TB07FTA|TB10FTA|z1000|Z99\ 2G|z99|z930|z999|z990|z909|Z919|z900|TOUCHPAD.*[78910]|Broncho.*(N701|N708|N802|a710)|Pantech.*P4100|\bN-06D|\bN-08D|T-Hub2|Android.*\bNabi|Playstation.*(Portable|Vita) Tablet;


	~*SM-N|Tapatalk|PDA|PPC|SAGEM|mmp|pocket|psp|symbian|Smartphone|smartfon|treo|up.browser|up.link|vodafone|wap|nokia|Series40|Series60|S60|SonyEricsson|N900|MAUI.*WAP.*Browser|LG-P500|iPhone.*Mobile|iPod|iTunes|BlackBerry|\bBB10\b|rim[0-9]+|HTC|HTC.*(Sensation|Evo|Vision|Explorer|6800|8100|8900|A7272|S510e|C110e|Legend|Desire|T8282)|APX515CKT|Qtek9090|APA9292KT|HD_mini|Sensation.*Z710e|PG86100|Z715e|Desire.*(A8181|HD)|ADR6200|ADR6425|001HT|Inspire\ 4G|Android.*\bEVO\b|Nexus\ One|Nexus\ S|Galaxy.*Nexus|Android.*Nexus.*Mobile|Dell.*Streak|Dell.*Aero|Dell.*Venue|DELL.*Venue\ Pro|Dell\ Flash|Dell\ Smoke|Dell\ Mini\ 3iX|XCD28|XCD35|\b001DL\b|\b101DL\b|\bGS01\b|sony|SonyEricsson|SonyEricssonLT15iv|LT18i|E10i|Asus.*Galaxy|PalmSource|Palm|Vertu|Vertu.*Ltd|Vertu.*Ascent|Vertu.*Ayxta|Vertu.*Constellation(F|Quest)?|Vertu.*Monika|Vertu.*Signature|IQ230|IQ444|IQ450|IQ440|IQ442|IQ441|IQ245|IQ256|IQ236|IQ255|IQ235|IQ245|IQ275|IQ240|IQ285|IQ280|IQ270|IQ260|IQ250|\b(SP-80|XT-930|SX-340|XT-930|SX-310|SP-360|SP60|SPT-800|SP-120|SPT-800|SP-140|SPX-5|SPX-8|SP-100|SPX-8|SPX-12)\b|PANTECH|IM-A|VEGA\ PTL21|PT003|P8010|ADR910L|P6030|P6020|P9070|P4100|P9060|P5000|CDM8992|TXT8045|ADR8995|IS11PT|P2030|P6010|P8000|PT002|IS06|CDM8999|P9050|PT001|TXT8040|P2020|P9020|P2000|P7040|P7000|C790|Samsung|BGT-S5230|GT-B2100|GT-B2700|GT-B2710|GT-B3210|GT-B3310|GT-B3410|GT-B3730|GT-B3740|GT-B5510|GT-B5512|GT-B5722|GT-B6520|GT-B7300|GT-B7320|GT-B7330|GT-B7350|GT-B7510|GT-B7722|GT-B7800|GT-C3|GT-C5010|GT-C5212|GT-C6620|GT-C6625|GT-C6712|GT-E|GT-I|GT-M3510|GT-M5650|GT-M7500|GT-M7600|GT-M7603|GT-M8800|GT-M8910|GT-N7000|GT-P6810|GT-P7100|GT-S|GT-S8530|GT-S8600|SCH-A310|SCH-A530|SCH-A570|SCH-A610|SCH-A630|SCH-A650|SCH-A790|SCH-A795|SCH-A850|SCH-A870|SCH-A890|SCH-A930|SCH-A950|SCH-A970|SCH-A990|SCH-I100|SCH-I110|SCH-I400|SCH-I405|SCH-I500|SCH-I510|SCH-I515|SCH-I600|SCH-I730|SCH-I760|SCH-I770|SCH-I830|SCH-I910|SCH-I920|SCH-LC11|SCH-N150|SCH-N300|SCH-R100|SCH-R300|SCH-R351|SCH-R4|SCH-T300|SCH-U|SCS-26UC|SGH-A|SGH-B|SGH-C|SGH-D307|SGH-D|SGH-D807|SGH-D980|SGH-E105|SGH-E200|SGH-E315|SGH-E316|SGH-E317|SGH-E335|SGH-E590|SGH-E635|SGH-E715|SGH-E890|SGH-F300|SGH-F480|SGH-I|SGH-J150|SGH-J200|SGH-L170|SGH-L700|SGH-M110|SGH-M150|SGH-M200|SGH-N|SGH-N7|SGH-P|SGH-Q105|SGH-R210|SGH-R220|SGH-R225|SGH-S105|SGH-S307|SGH-T|SGH-U|SGH-V|SGH-X|SGH-Z130|SGH-Z150|SGH-Z170|SGH-ZX10|SGH-ZX20|SHW-M110|SPH-A|SPH-D600|SPH-D700|SPH-D710|SPH-D720|SPH-I300|SPH-I325|SPH-I330|SPH-I350|SPH-I500|SPH-I600|SPH-I700|SPH-L700|SPH-M|SPH-N100|SPH-N200|SPH-N240|SPH-N300|SPH-N400|SPH-Z400|SWC-E100|SCH-i909|GT-N7100|GT-N8010|Motorola|\bDroid\b.*Build|DROIDX|Android.*Xoom|HRI39|MOT-|A1260|A1680|A555|A853|A855|A953|A955|A956|Motorola.*ELECTRIFY|Motorola.*i1|i867|i940|MB200|MB300|MB501|MB502|MB508|MB511|MB520|MB525|MB526|MB611|MB612|MB632|MB810|MB855|MB860|MB861|MB865|MB870|ME501|ME502|ME511|ME525|ME600|ME632|ME722|ME811|ME860|ME863|ME865|MT620|MT710|MT716|MT720|MT810|MT870|MT917|Motorola.*TITANIUM|WX435|WX445|XT3|XT502|XT530|XT531|XT532|XT535|XT6|XT7|XT8|XT9 Mobile;

	
} 




   

    location / {
	add_header X-Mobile-Device $mobile_device;
        root   /data/www/www.example.com;
        index  index.html index.htm index.php info.php;
	try_files $uri $uri/ /index.php?$args;
	
	
    }

    location ~ \.php$ {
        root           /data/www/www.example.com;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
	fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include        fastcgi_params;
	
	fastcgi_cache_key "$request_method$request_uri$http_user_agent";
        fastcgi_cache_valid 404 30m;
        fastcgi_cache_valid 200 10m;
        fastcgi_max_temp_file_size 1M;
        fastcgi_cache_use_stale updating;
        fastcgi_pass_header Set-Cookie;
        fastcgi_pass_header Cookie;
        fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

	fastcgi_param PHP_VALUE "MY-Device=$mobile_device";	

	add_header Cache-Control: no-transform,public,max-age=300,s-maxage=900;		
	add_header N-Cache $upstream_cache_status;
    }

So here you will recieve php value or x-cache as My-Device.

Nginx + php-fpm + Page caching

This is an example of nginx configuration default.conf:

Usages: Fastcgi, php-fpm, Page caching


fastcgi_cache_path /etc/nginx/cache levels=1:2 keys_zone=MYAPP:100m inactive=60m;
fastcgi_temp_path /etc/nginx/cache/temp;

#
# The default server
#
server {
listen 80;
server_name example.com www.example.com;
#autoindex on;
server_tokens off;

#Cache everything by default
set $no_cache 0;

#Don't cache POST requests
if ($request_method = POST)
{
set $no_cache 1;
}

##Don't cache if the URL contains a query string
if ($query_string != "")
{
set $no_cache 1;
}

#Don't cache the following URLs
if ($request_uri ~* "/(wp-admin/|wp-login.php)")
{
set $no_cache 1;
}

gzip on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
#gzip_vary on;
gzip_http_version 1.1;
gzip_comp_level 1;
gzip_proxied any;
gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js ;
gzip_buffers 16 8k;

location / {
root /data/www/www.example.com;
index index.html index.htm index.php info.php;
try_files $uri $uri/ /index.php?$args;

location ~ \.php$ {
root /data/www/www.example.com;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_cache MYAPP;
fastcgi_no_cache $no_cache;
fastcgi_cache_bypass $no_cache;
fastcgi_cache_key "$request_method$request_uri$http_user_agent";

fastcgi_cache_valid 404 30m;
fastcgi_cache_valid 200 10m;
fastcgi_max_temp_file_size 1M;
fastcgi_cache_use_stale updating;
fastcgi_pass_header Set-Cookie;
fastcgi_pass_header Cookie;
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

add_header Cache-Control: no-transform,public,max-age=300,s-maxage=900;
add_header N-Cache $upstream_cache_status;
}

# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
location ~ /\.ht {
deny all;
}
}

General Nginx Configuration

user nginx;
worker_processes 4; #What is Worker_processes?

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

worker_rlimit_nofile 10000; #What is worker_rlimit_nofile?
events {
worker_connections 9000;#What is worker_connection?
multi_accept on;
use epoll;# Connection processing

}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format main ‘$remote_addr – $remote_user [$time_local] “$request” ‘
‘$status $upstream_cache_status $body_bytes_sent “$http_referer” ‘
‘”$http_user_agent” “$http_x_forwarded_for”‘;

access_log /var/log/nginx/access.log main;

server_tokens off;
sendfile on;
tcp_nopush on;
tcp_nodelay off;
keepalive_timeout 5;

include /etc/nginx/conf.d/*.conf;
}

nginx redirect loop

The page isn’t redirecting properly
Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

I have noticed that this could happen in many diff reasons.

1) Nginx Configuration
e.g. http://stackoverflow.com/questions/4616521/nginx-configuration-leads-to-endless-redirect-loop

2) can be when you dont have extforward trusted on backend server which is used from nginx(as reverse proxy).

e.g.
extforward.headers = (“X-Forwarded-For”, “Forwarded-For”)

extforward.forwarder = (
“192.168.1.2” => “trust”
)

Nginx SSL Termination, Varnish

If you are looking for ssl termination on varnish, i would assume you have enough knowledge of configuring varnish and looking into this you must be able to configure ssl termination including Client-IP forwarding.

Default location with 444 error would make sense to forward any unwated traffic, you may want to google it around if its not already in www.ashishnepal.com 😉

server {
### server port and name ###
listen 192.168.1.24:443;
ssl on;
server_name www.tikejhya.com;

### SSL log files ###
access_log logs/ssl-access.log;
error_log logs/ssl-error.log;

### SSL cert files ###
ssl_certificate /var/certs/tikejhya.com.crt;
ssl_certificate_key /var/certs/tikejhya.com.key;

ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
keepalive_timeout 60;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

location / {
# Pass the request on to Varnish.
proxy_pass http://127.0.0.1;

# Pass a bunch of headers to the downstream server, so they’ll know what’s going on.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

# Most web apps can be configured to read this header and understand that the current session is actually HTTPS.
proxy_set_header X-Forwarded-Proto https;
# We expect the downsteam servers to redirect to the right hostname, so don’t do any rewrites here.
proxy_redirect off;
}
}

Powered by WordPress & Theme by Anders Norén