UW - Madison's Load Balancing capabilities

The Nortel 3408 load balancers were replaced with dual Citrix Netscaler 17000s back in 2010. The Netscaler's (both production and test) are setup in a high availability (HA) pair which we've tested 1-2 second failover. We currently have one appliance at Walnut Street and the other in the Computer Science datacenter. This mirrors how routing is also setup. If a datacenter experiences issues, the load balancers and routers are setup to fail to the backup site. As of January 2017, new Citrix Netscaler load balancers were being used. These new Netscalers are SDX14020, split between CSSC and Animal and connected to the DDN core routers. The SDX14020 has the ability allocate virtual load balancers called VPX's. Both Production and Test environments will run on this hardware in separate VPXs.

Quick Links:

Load Balancing Metrics
Health Check Monitor/Probe Descriptions
Persistence Descriptions

The following is a general list of features that the new load balancers support today.

Administrative

Access

The Citrix Netscaler has a web interface for web management. We allow App and Server admins access to this interface in a limited capacity. Today you'll be able to disable/enable servers and services, and replace SSL certificates that belong to you only.

Please create an WiscIT ticket (send to Network Services) if you are interested in this feature. Please include your Name, Phone #, Email, NetID, Department Name, Virtual IPs(VIPs) you manage the service for, and your managers name.

Statistics

There are built in statistics on the Citrix Netscaler down to a per service level but can take time to setup and manage. It also does not keep stats on all items.  It seems to be limited to 256 objects.  DoIT Network Services has setup automatic stats collecting which can be viewed via https://stats.net.wisc.edu/layer4/. Every morning our stats collector is restarted to go fetch any new objects that may have been configured the day before.

Please create an Infra call (send to Network Services) if you'd like additional stats views.

Monitoring

Network Services initially setup VIP and service monitoring, however the NOC requested we stop since they already had a way to monitor these objects.

Application Switching and Traffic Management Features

IPv6 to IPv4 / IPv4 to IPv6 / IPv6 to IPv6

Translating IPv6 virtual IPs to IPv4 servers and vise-versa is now available. Allowing the UW to provide IPv6 capable services using existing infrastructure and a smoother transition to IPv6. IPv6 to IPv4 and IPv4 to IPv6 utilizes IP proxying between the two IP stacks. This means that client requests will appear to be coming from the Citrix Netscaler's IP address. For specific services, like HTTP, we can inject the clients IP address into the X-Forwarded-For(XFF) header.

Please use http://platform.doit.wisc.edu/ to create a request.

SSL Offloading

Transparently offloads SSL encryption and decryption from Web servers, freeing server resources to service content requests. SSL places a heavy burden on an application's performance and can render many optimization measures ineffective. SSL offload and acceleration allow all the benefits of Citrix Request Switching technology to be applied to SSL traffic, ensuring secure delivery of web applications without degrading end-user performance.

Please use http://platform.doit.wisc.edu/ to create a request.

Access Control Lists

Compares incoming packets to the Access Control Lists (ACLs). If a packet matches an ACL rule, the action specified in the rule is applied to the packet. Otherwise, the default action (ALLOW) is applied and the packet is processed normally by the system.

For the UW implementation, we plan to utilize the campus firewall service instead of using the Netscaler ACL feature.

Load Balancing

Manages traffic at the request level, resulting in more uniform traffic distribution across servers, as compared to the conventional approach to distributing connections among servers. Load balancing decisions are based on a variety of policies, including round robin, least connections, weighted least bandwidth, weighted least packets, minimum response time, and hashing based on URL, domain source IP or destination IP. Click here for a more detailed list. Both the TCP and UDP protocols are supported, so the NetScaler can load balance all traffic that uses those protocols as the underlying carrier (for example, HTTP, HTTPS, UDP, DNS, FTP, and NNTP). In addition, the NetScaler can maintain session persistence based on source IP, cookie, server, group, SSL session, SIP CALLID, token-based, JSESSIONID, and other advanced routines. Click Netscaler9.2_persistence_types.pdf for more details. It allows users to apply custom Extended Content Verification (ECV) to servers, caches, firewalls and other infrastructure devices to ensure that these systems are functioning properly and are providing the right content to users. It can also perform health checks using ping, TCP, or HTTP URL, and the user can create monitors based on Perl scripts. Details on health check monitors/probes can be found in NS-Health-Check-Probes.pdf.

As an FYI, we can provide enable/disable/monitoring access to the services you are responsible for.

Please use http://platform.doit.wisc.edu/ to create a request.

Content Switching

Determines which server is best able to respond and switches individual content requests to that server. Site rules can be configured based on URL and any combination of HTTP headers. This allows switching decisions to be based on user and device characteristics such as who the user is, what type of agent is being used, and what content the user requested.

Please use http://platform.doit.wisc.edu/ to create a request.

Global Server Load Balancing (GSLB)

Extends the traffic management capabilities of a NetScaler to include distributed Internet sites and global enterprises. Whether installations are spread across multiple network locations or multiple clusters in a single location, the NetScaler maintains availability and distributes traffic across them. It makes intelligent DNS decisions to prevent users from being sent to a site that is down or overloaded. When the proximity-based GSLB method is enabled, the NetScaler can make load balancing decisions based on the proximity of the client’s local DNS server (LDNS) in relation to different sites. The main benefit of the proximity-based GSLB method is faster response time resulting from the selection of the closest available site.

At the UW we won't be using this feature today since we have datacenters in different locations.

Dynamic Routing

Enables routers to obtain topology information, routes, and IP addresses from neighboring routers automatically. When dynamic routing is enabled, the corresponding routing process listens to route updates and advertises routes.

At the UW we won't be using this feature today but will instead use the routing capabilities of our backbone and edge routers. Also we are running the Netscaler as a bridge instead of a router today, allowing the Netscaler do what it does best and our routers to do what they do best.

Application Acceleration Features

AppCompress

Provides transparent compression for HTML and text files using the GZip compression protocol. The typical 4:1 compression ratio yields up to 50% reduction in bandwidth requirements out of the data center. This also results in significantly improved end-user response time by reducing the amount of data that must be delivered to the user’s browser.

All services today have this enabled by default.

Please use http://platform.doit.wisc.edu/ to create a request to change the default behavior.

Cache Redirection

Manages the flow of traffic to a reverse proxy, transparent proxy, or forward proxy cache farm. Inspects all requests, and identifies non-cacheable requests and sends them directly to the origin servers over persistent connections. By intelligently redirecting non-cacheable requests back to the origin web servers, the NetScaler frees cache resources and increases cache hit rates while reducing overall bandwidth consumption and response delays for these requests. At the UW we don't have a cache farm today but if there is enough interest we can look into, especially if the following feature does not meet our needs.

AppCache

Helps optimize Web content and application data delivery by providing a fast in-memory HTTP/1.1 and HTTP/1.0 compliant Web caching for both static and dynamic content. This on-board cache stores the results of incoming application requests even when an incoming request is secured or the data compressed, and then reuses the data to fulfill subsequent requests for the same information. By serving data directly from the on-board cache, the NetScaler can reduce page regeneration times by eliminating the need to funnel static and dynamic content requests to the server.

Please use http://platform.doit.wisc.edu/ to create a request.

TCP Buffering

Buffers the server’s response and delivers it to the client at the client’s speed, thus offloading the server faster and thereby improving the performance of Web sites.  This may vary by service type.

Please use http://platform.doit.wisc.edu/ to create a request.

Application Security and Firewall Features

Denial of Service Attack (DoS) Defense

Detects and stops malicious distributed denial-of-service (DDoS) attacks and other types of malicious attacks before they reach your servers, preventing them from affecting network and application performance. The NetScaler identifies legitimate clients and elevates their priority, leaving suspect clients unable to consume a disproportionate percentage of resources and cripple your site. A NetScaler provides application-level protection from the following types of malicious attacks:
  • SYN flood attacks
  • Pipeline attacks
  • Teardop attacks
  • Land attacks
  • Fraggle attacks
  • Zombie connection attacks
The NetScaler aggressively defends against these types of attacks by preventing the allocation of server resources for these connections. This insulates servers from the overwhelming flood of packets associated with these events. The NetScaler also protects network resources from ICMP based attacks by using ICMP rate limiting and aggressive ICMP packet inspection. It performs strong IP reassembly, drops a variety of suspicious and malformed packets, and applies Access Control Lists (ACLs) to site traffic for further protection.

In most cases the campus firewall service will help mitigate the attacks mentioned above.

Content Filtering

Provides protection from malicious attacks for web sites at the Layer 7 level. The NetScaler inspects each incoming request according to user-configured rules based on HTTP headers, and performs the action the user configured. Actions can include resetting the connection, dropping the request, or sending an error message to the user’s browser. This allows the NetScaler to screen unwanted requests and reduces your servers’ exposure to attacks. This feature can also analyze HTTP GET and POST requests and filter out known bad signatures, allowing it to defend your servers against HTTP-based attacks such as variants of the Nimda and Code Red viruses.

Please use http://platform.doit.wisc.edu/ to create a request.

Responder

Functions like an advanced content filter that can be used to generate responses from the Netscaler system to the client. Some common uses of this feature are generation of redirect responses, user defined responses or resets. Deals with only the request side of the system unlike the content filtering feature which deals with requests just before they are about to be sent to the back end servers.

An example of this feature, we are able to redirect port 80 traffic to port 443 without having to send the port 80 traffic to the server first. Instead the port redirection occurred at the VIP level.

This is a little different than port redirection, which is also supported. The difference between the two is that the client doesn't know that the VIP service port is different than the ports on the servers. With the responder feature, the Netscaler is telling the client to redirect somewhere else.

Please use http://platform.doit.wisc.edu/ to create a request.

HTTP Rewrite

Modifies HTTP headers and body text. You can use it to add HTTP headers to an HTTP request or response, make modifications to individual HTTP headers, or delete HTTP headers. It also lets you modify the HTTP body in requests and responses.

When the NetScaler receives a request or sends a response, it checks for rewrite rules, and if applicable rules exist, it applies them to the request or response before passing it on to the Web server or client computer.

Please use http://platform.doit.wisc.edu/ to create a request.

Priority Queuing

Prioritizes user requests to ensure that the most important traffic is serviced first during surges in request volume. You can establish priority based on request URLs, cookies, or a variety of other factors. The NetScaler places requests in a three-tier queue based on their configured priority, enabling business-critical transactions to flow smoothly even during surges or site attacks.

Since many services will be using the Netscalers at the UW, if this is requested, a group (perhaps E-Infrastructure) will have to decide what services are more important over others.

Surge Protection

Regulates the flow of user requests to servers and controls the number of users that can simultaneously access the resources on the servers, queuing any additional requests once your servers have reached their capacity. By controlling the rate at which connections can be established, the NetScaler blocks surges in requests from being passed on to your servers, thus preventing site overload.

Please use http://platform.doit.wisc.edu/ to create a request.

Access Gateway

Securely delivers any application with policy-based SmartAccess control. Users can obtain easy-to use secure access to all of the enterprise applications and data they need to be productive. IT organizations can cost-effectively extend access to applications outside the data center while maintaining strict control through SmartAccess application-level policies. IT organizations are empowered to cost-effectively meet the demands of all workers, deliver flexible working options, and implement business continuity while ensuring the highest-level of information security and reducing support calls.

At the UW we have no plans to implement this feature at the Netscaler level but we do have Departmental VPN and  the campus VPN service that allows off-site users access to campus resources.

Application Firewall

Protects applications from misuse by hackers and malware, such as cross site scripting attacks, buffer overflow attacks, SQL injection attacks, and forceful browsing, by filtering traffic between each protected Web server and users that connect to any Web site on that Web server. The Application Firewall examines all traffic for evidence of attacks on Web server security or misuse of Web server resources, and takes the appropriate action to prevent these attacks from succeeding.

Please use http://platform.doit.wisc.edu/ to create a request.

Application Visibility Feature

EdgeSight for NetScaler

Support for application performance monitoring based on end user experience. This solution leverages the HTML injection feature to obtain various time values, which are used by EdgeSight server for analysis and report generation. EdgeSight for NetScaler provides a way to monitor the performance benefits of a NetScaler and determine potential bottlenecks in a network.

If there is interest at the UW for this feature we'll need to setup a project and determine how it should be managed and funded.

If you are interested in this feature, please create an Infra call and have it routed to DoIT-Campus Network Services.