| View previous topic :: View next topic |
| Author |
Message |
stig
Vyatta Employee
Joined: 21 Feb 2008
Posts: 1282
Location: silicon valley
|
 Posted: Tue Nov 17, 2009 4:44 pm Post subject: netflow/sflow in vyatta vc6 alpha (a quick howto) |
 |
|
The V6 alpha (see: http://www.vyatta.org/forum/viewtopic.php?t=3033) supports collecting flows and exporting to either a netflow or sflow collector. (note: for those wondering, the underlying package is pmacct).
To configure:
A) If you just want to keep the flows in memory, just add the interfaces you're interested in collecting:
| Code: | vyatta@R1:~$ configure
[edit]
vyatta@R1# set system flow-accounting interface eth ?
eth0 eth1
[edit]
vyatta@R1# set system flow-accounting interface eth0
[edit]
vyatta@R1# set system flow-accounting interface eth1
[edit]
vyatta@R1# commit
Starting [eth1] accounting
Starting [eth0] accounting
[edit]
vyatta@R1# save
Saving configuration to '/opt/vyatta/etc/config/config.boot'...
Done
[edit]
vyatta@R1# exit
exit
vyatta@R1:~$ |
B) to also export netflow v5 to a netflow collector at 172.16.117.25 port 2055:
| Code: | vyatta@R1:~$ configure
[edit]
vyatta@R1# set system flow-accounting netflow version 5
[edit]
vyatta@R1# set system flow-accounting netflow server 17.16.117.25
[edit]
vyatta@R1# set system flow-accounting netflow server 17.16.117.25 port 2055
[edit]
vyatta@R1# commit
Stopping [eth1] accounting
Starting [eth1] accounting
Stopping [eth0] accounting
Starting [eth0] accounting
[edit]
vyatta@R1# save
Saving configuration to '/opt/vyatta/etc/config/config.boot'...
Done
[edit]
vyatta@R1# exit
exit
vyatta@R1:~$ |
For the in memory table there are the following operational commands:
| Code: | vyatta@R1:~$ show flow-accounting interface eth0
flow-accounting for [eth0]
Src Addr Dst Addr Sport Dport Proto Packets Bytes Flows
10.1.0.14 224.0.0.18 0 0 vrrp 233 9320 1
69.59.150.135 10.1.0.90 123 123 udp 12 912 2
76.74.103.45 10.1.0.90 0 0 icmp 2 168 1
10.0.0.30 10.1.0.90 53 40089 udp 1 133 1
10.0.0.30 10.1.0.90 53 36641 udp 1 133 1
10.0.0.30 10.1.0.90 53 51363 udp 1 106 1
Total entries: 6
Total flows : 7
Total pkts : 250
Total bytes : 10,772
|
To clear flows counters from the memory table:
| Code: | vyatta@R1:~$ clear flow-accounting interface eth0 counters
clearings flow-accounting for [eth0]
OK: Clearing stats. |
To show the top N flows for an interface:
| Code: | vyatta@R1:~$ show flow-accounting interface eth0 top 10
flow-accounting for [eth0]
Src Addr Dst Addr Sport Dport Proto Packets Bytes Flows
4.23.63.126 10.1.0.90 80 4225 tcp 621 852482 1
84.40.22.66 10.1.0.90 80 3886 tcp 326 416490 1
84.40.22.66 10.1.0.90 80 3887 tcp 274 343532 1
72.246.87.172 10.1.0.90 80 3895 tcp 177 251417 1
4.23.63.126 10.1.0.90 80 4227 tcp 129 158823 1
207.123.55.126 10.1.0.90 80 4602 tcp 96 90493 1
4.23.63.126 10.1.0.90 80 4228 tcp 58 76388 1
207.123.55.126 10.1.0.90 80 4603 tcp 84 72228 1
216.34.181.45 10.1.0.90 80 1666 tcp 34 43324 1
74.125.19.167 10.1.0.90 80 3692 tcp 34 43023 1
Total entries: 10
Total flows : 10
Total pkts : 1,833
Total bytes : 2,348,200 |
To show flows for a given host:
| Code: | vyatta@R1:~$ show flow-accounting interface eth0 host 84.40.22.66
Src Addr Dst Addr Sport Dport Proto Packets Bytes Flows
84.40.22.66 10.1.0.90 80 3886 tcp 326 416490 1
84.40.22.66 10.1.0.90 80 3887 tcp 274 343532 1
84.40.22.66 10.1.0.90 21 1561 tcp 2 343 1
84.40.22.66 10.1.0.90 0 0 icmp 3 252 1
Total entries: 4
Total flows : 4
Total pkts : 605
Total bytes : 760,617
|
To flows for a given port:
| Code: | vyatta@R1:~$ show flow-accounting interface eth0 port 123
Src Addr Dst Addr Sport Dport Proto Packets Bytes Flows
69.59.150.135 10.1.0.90 123 123 udp 10 760 10
69.59.150.135 10.1.0.90 123 123 udp 10 760 10
Total entries: 2
Total flows : 20
Total pkts : 20
Total bytes : 1,520
|
One caveat: the current package uses libpcap which probably isn't recommended for high speed interfaces. The plan is to switch to pmacct's ULOG support in one of the next alpha/beta builds, but we wanted to get some early feedback with the existing code. |
|
| Back to top |
|
 |
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Mon Dec 14, 2009 4:44 am Post subject: |
|
|
Thanks a lot for this great feature.
Configuration was very easy... I am using the VC6 alpha OVF on ESXi 4.0 server. CPU load is very low, no clear issues what so ever.
However, I am trying Scrutinizer as a netflow collector and it said that:
"Scrutinizer expects NetFlow & IPFIX to be exported by the router every 60 seconds to provide the front end with 1 minute granularity. When flows are exported in greater than 60 second intervals, spikes well above 100% utilization can occur.
The following devices are exporting conversations in an interval longer than 1 minute:
TELNET TO DEVICE configure NetFlow via SNMP 192.168.100.254 (192.168.100.254) - longest export was 590 seconds"
Any idea why this could be happening??
Do you recommend any good free Netflow Collector?? |
|
| Back to top |
|
|
stig
Vyatta Employee
Joined: 21 Feb 2008
Posts: 1282
Location: silicon valley
|
| Posted: Mon Dec 14, 2009 9:27 am Post subject: |
|
|
| adabbas wrote: | However, I am trying Scrutinizer as a netflow collector and it said that:
"Scrutinizer expects NetFlow & IPFIX to be exported by the router every 60 seconds to provide the front end with 1 minute granularity. When flows are exported in greater than 60 second intervals, spikes well above 100% utilization can occur.
The following devices are exporting conversations in an interval longer than 1 minute:
TELNET TO DEVICE configure NetFlow via SNMP 192.168.100.254 (192.168.100.254) - longest export was 590 seconds"
Any idea why this could be happening?? |
Try decreasing the timeout values:
| Code: | vyatta@R1# set system flow-accounting netflow timeout ?
Possible completions:
expiry-interval
Set expiry scan interval
flow-generic Set flow generic timeout value
icmp Set icmp timeout value
max-active-life
Set max active timeout value
tcp-fin Set tcp finish timeout value
tcp-generic Set tcp generic timeout value
tcp-rst Set tcp reset timeout value
udp Set udp timeout value |
Netflow initially was a switching technology more than an accounting technology, so the flow timers were set to expire slowly. Some netflow collectors want more of a real time view of the flows, so they would like the flows to expire very quickly. For example, say you have a long file transfer going, for the netflow collector to "see" that it needs the flow to exported several times a minute rather than at the end of the transfer.
| adabbas wrote: | | Do you recommend any good free Netflow Collector?? |
Good question. I would be nice if people would list what they're using as a flow collector here and mention the pros/cons. I've tried several and haven't really found one that meets all my needs. I liked the nice graphs that Scrutinizer has, and it was very easy to install, but the free version has a 1 day limit on data collection. To me that is a severe limitation (of course that's just my personal opinion). |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Mon Dec 14, 2009 1:20 pm Post subject: |
|
|
It seems that I am facing a really big issue with netflow that seems to be a result of my network topology:
Scrutinizer seems to think that I have only one Interface (eth0), but in fact I have 4 virtual interfaces eth0.99, eth0.100, eth0.101 and eth0.102 where eth0.99 and eth0.100 used for LAN and eth0.101 and eth0.102 used for WAN.
As a result: It always reports the same amount of traffic as inbound and outbound without distinguishing which traffic is coming from which WAN link and to which internal network it is going.
Moreover (and more importantly) I am unable to see which of my internal hosts is sucking the most of the Internet bandwidth because it can only show my internal hosts as a source for traffic and never as a destination for it. All destinations listed are IP addresses on the Internet. So I can only see upload traffic but not download traffic!!
This does not change if I try to view inbound or outbound traffic. Maybe because the netflow collector (or the exporter) views all virtual interfaces as one interface (eth0).
I can try to add three more NICs to see if this odd behavior changes or (much easier) try another collector to see if it is smarter… but I am reporting this now for the vyatta people to make sure that the problem is not with the exporter as I am sure that other users/customers may want to moniter their router-on-stick devices.
Last edited by adabbas on Mon Dec 14, 2009 3:34 pm; edited 2 times in total |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Mon Dec 14, 2009 2:27 pm Post subject: |
|
|
By the way, I've changed "expiry-interval" to 15 sec and still getting the same error but now Scrutinizer reports that the "longest export was 244 seconds" only 
It could be a side effect of hosting either Vyatta on ESXi or (more likely) Scrutinizer Red Hat appliance on VMware workstation. I've read before the Linux in general has time keeping problems when virtualized, did not see it on ESXi yet (my Server CPUs is extremely underutilized) and that is why I think that the Scrutinizer appliance has a timing issue.
Problem is that both VMs are reporting the correct time, so I can’t the to the time reading and say “ahha, I blame you" 
Anyway, this is not a big issue; my problem above in regard to VLANs is more of an issue that needs more investigating as it could be a showstopper indeed. |
|
| Back to top |
|
|
stig
Vyatta Employee
Joined: 21 Feb 2008
Posts: 1282
Location: silicon valley
|
| Posted: Mon Dec 14, 2009 4:18 pm Post subject: |
|
|
| adabbas wrote: | By the way, I've changed "expiry-interval" to 15 sec and still getting the same error but now Scrutinizer reports that the "longest export was 244 seconds" only
It could be a side effect of hosting either Vyatta on ESXi or (more likely) Scrutinizer Red Hat appliance on VMware workstation. I've read before the Linux in general has time keeping problems when virtualized, did not see it on ESXi yet (my Server CPUs is extremely underutilized) and that is why I think that the Scrutinizer appliance has a timing issue. |
I did my testing from vyatta running on virtual machines and didn't have a problem with timing. Which version of netflow are you using? It defaults to version 5 with should be setting the correct input interface.
BTW, In the current development branch I have changed the implementation to use ULOG instead of libpcap, which should be a performance enhancement. Hopefully there will be another alpha build soon. |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Mon Dec 14, 2009 4:38 pm Post subject: |
|
|
| stig wrote: | | Which version of netflow are you using? It defaults to version 5 with should be setting the correct input interface. |
Maybe that is it, I've set to version 9... I will see if changing it to 5 changes things.
| stig wrote: | | should be setting the correct input interface. |
Do you mean that it will not treat all my 4 vif as one?? As this is a much bigger issue to me. I will test it in an hour.
| stig wrote: | | BTW, In the current development branch I have changed the implementation to use ULOG instead of libpcap, which should be a performance enhancement. Hopefully there will be another alpha build soon. |
That sure is a good thing for all of us, and I do not want to sound unthankful, but higher performance does not really affect many SMBs. My router used to utilize about 15MHz with VC5, now it is utilizing around 70MHz with netflow on  so Vyatta's performance is already very impressive.
Last edited by adabbas on Tue Dec 15, 2009 1:59 pm; edited 1 time in total |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Mon Dec 14, 2009 6:10 pm Post subject: |
|
|
Thanks, after changing to netflow version 5, I can see 4 Interfaces.
But still I cannot see any traffic with my Internal hosts as destination Not on the inbound or the outbound
Now this is becoming very confusing. |
|
| Back to top |
|
|
stig
Vyatta Employee
Joined: 21 Feb 2008
Posts: 1282
Location: silicon valley
|
| Posted: Mon Dec 14, 2009 6:23 pm Post subject: |
|
|
| adabbas wrote: | | Thanks, after changing to netflow version 5, I can see 4 Interfaces. |
netflow version 9 wasn't properly setting the input interface, that's fixed now and should work in the next alpha release.
| adabbas wrote: |
But still I cannot see any traffic with my Internal hosts as destination Not on the inbound or the outbound . |
Hmm, odd. Does it show up in the memory table (show flow-accounting interface <>) ? Also note that it's only the input traffic of an interface (otherwise you'd get double accounting). |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Mon Dec 14, 2009 7:18 pm Post subject: |
|
|
| stig wrote: | | adabbas wrote: |
But still I cannot see any traffic with my Internal hosts as destination Not on the inbound or the outbound . |
Hmm, odd. Does it show up in the memory table (show flow-accounting interface <>) ? |
Not even there.
If I look at eth0.100 which is the internal LAN all destinations are on the Internet and all sources are local hosts (I want to see traffic going to local hosts not from Local hosts).
If I look at eth0.101 where the 1st ADSL modem is connected I see traffic form the Internet (many real IPs) to 192.168.101.250 (Vyatta's static IP on this VLAN).
If I look at eth0.101 where the 2nd ADSL modem is connected I see traffic form the Internet (many real IPs) to 192.168.102.250 (Vyatta's static IP on this VLAN).
But I can never tell what traffic is coming from the Internet to a local host, although I can easily see what traffic is going from local hosts to the Internet
| stig wrote: | | Also note that it's only the input traffic of an interface (otherwise you'd get double accounting). |
Not sure I understand this correctly. The way I see it If I want to look at traffic going to my internal hosts then I should be looking at traffic going out of (or traversing) the router eth0.100 interface to local VLAN... am I wrong?? Or at least traffic coming inbound to my WAN links but shown destined to my local hosts and not to Vyatta WAN IPs.
I am using WLB, I wonder if it act the same under Natting 
Do you need me to post all my config?? |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Tue Dec 15, 2009 8:38 am Post subject: |
|
|
Dear Stig,
Any update about this issue?? Is net flow working ok with WLB or Natting??
Can it distinguish between traffic destined to the router and traffic that is traversing the router to internal hosts?? |
|
| Back to top |
|
|
stig
Vyatta Employee
Joined: 21 Feb 2008
Posts: 1282
Location: silicon valley
|
| Posted: Tue Dec 15, 2009 8:46 am Post subject: |
|
|
| adabbas wrote: | Dear Stig,
Any update about this issue?? Is net flow working ok with WLB or Natting??
Can it distinguish between traffic destined to the router and traffic that is traversing the router to internal hosts?? |
Given that libpcap happens at packet capture, I'd imagine it gets the packets before any NAT has occurred. The new ULOG support hooks into netfilter, so we could choose to hook it in early or after firewall, nat. |
|
| Back to top |
|
|
adabbas
Super User
Joined: 23 Aug 2008
Posts: 259
|
| Posted: Tue Dec 15, 2009 9:23 am Post subject: |
|
|
So hooking it in after nat will solve the issue??
I hope this also applies to WLB, does not it?
PS so there is a problem, it was not my fault |
|
| Back to top |
|
|
stig
Vyatta Employee
Joined: 21 Feb 2008
Posts: 1282
Location: silicon valley
|
| Posted: Tue Dec 15, 2009 11:31 am Post subject: |
|
|
| adabbas wrote: | | So hooking it in after nat will solve the issue?? |
I guess we'll find out when the next vc6 alpha is released (or you can build the package yourself). |
|
| Back to top |
|
|
jwilling
Active Member
Joined: 27 Feb 2008
Posts: 22
|
| Posted: Fri Dec 18, 2009 2:16 pm Post subject: No accounting on second interface? |
|
|
Well... have flow-accounting configured pretty much as outlined in the example, and the system initially reports that it has started accounting on both interfaces (eth0, eth1), but it is not reporting on the eth1 interface to the collector and returns the following when queried locally:
- - -
######:/opt/vyatta/etc/config# show flow-accounting interface eth1
flow-accounting for [eth1]
Src Addr Dst Addr Sport Dport Proto Packets Bytes Flows
Total entries: 0
Total flows : 0
Total pkts : 0
Total bytes : 0
- - -
Any thoughts/hints? |
|
| Back to top |
|
|
|
Forum Subscriptions
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
