DaBeach

Ok, I like this. Here is a modifed windows batch file that will process "access.log" in the same directory that the batch is run in but uses a menu to select either a particular ip or all ip's into seperate files. @ECHO OFF :start CLS ECHO 1. 172.16.10.102 ECHO 2. 172.16.10.104 ECHO 3. 172.16.10.162 ECHO 4. PARSE ALL OF THE ABOVE ECHO. ECHO 5. EXIT ECHO. set choice= set /p choice= if not '%choice%'=='' set choice=%choice:~0,1% if '%choice%'=='1' goto 102 if '%choice%'=='2' goto 104 if '%choice%'=='3' goto 162 if '%choice%'=='4' goto ALL if '%choice%'=='5' goto EXIT ECHO. ECHO "%choice%" INVALID Choice, Please try again ECHO. ECHO. ECHO. GOTO START :102 ECHO PARSING 102 Find /I "172.16.10.102" access.log > ACCESS_LOG_102.txt start notepad.exe ACCESS_LOG_102.txt GOTO EXIT :104 ECHO PARSING 104 Find /I "172.16.10.104" access.log > ACCESS_LOG_104.txt start notepad.exe ACCESS_LOG_104.txt GOTO EXIT :162 ECHO PARSING 162 Find /I "172.16.10.162" access.log > ACCESS_LOG_162.txt start notepad.exe ACCESS_LOG_162.txt GOTO EXIT :ALL ECHO PARSING ALL Find /I "172.16.10.102" access.log > ACCESS_LOG_102.txt Find /I "172.16.10.104" access.log > ACCESS_LOG_104.txt Find /I "172.16.10.162" access.log > ACCESS_LOG_162.txt start notepad.exe ACCESS_LOG_102.txt start notepad.exe ACCESS_LOG_104.txt start notepad.exe ACCESS_LOG_162.txt CLS :EXIT Now, is there a way to run a windows batch file that would access the routers squid access.log and either parse it and place the data on the windows machine or copy the router file to the windows machine to process it? With pfSense I would have to go into the router and via the WWW GUI run the edit file option and rummage through the folder until I found the access file, I would then have to copy and past and save it to the windows machine. I would think there would be an automated way to do this. Any suggestions?

Hmm. Since this board is a Micro ATX, I guess I would stick with the unit I have (the 3ghz). I considered keeping the spare system as a router but did very much like the idea of the smaller form factor that Darren is building. Now you mentioned installing additional HDD's to use the system as a firewall and a NAS device. How would this be done as would it not require 2 OS's? Fill me in on this one. As one of the reasons I liked the Micro ITX unit was because I could stack them or perhaps put them in cases and use the additional units for media or storage.

Anyone have any recommendations on which Mini ITX MB & CPU to choose for a pfSense router install? What would you buy and why?

Ok. I checked the cache directory and found many sub directories from 00 through 09, I checked a few say 00 and found again many sub directories 00 through ff, then the 00 finding 2.94kb document as filename 00000000. Would I not find in these directories the actual files downloaded? If say most websites visited are dynamic sites I would expect not to find html but what about jpg or zip or exe etc...? Additionally I was browsing the logs and did like the ability to view the access.log which showed what seems to be all http with links to pics downloaded etc including access to dynamic page addresses. However I am disappointed that I would have to export this out into say a database to analyze what IP did what. I had tried a squid add on package that was supposed to view the log and format it up however, it is not currently setup to work with pfSense snapshots. Would there be another way to say access the router box with putty and grab out the access.log file by command line and have another program open and format it at the click of an icon? I am not sure if say Wireshark could do this I have not yet tried. And by the way thanks for your help and input on my posts.

Currently I am running pfSense with the Squid proxy system setup. If I understand this proxy system correct it caches every web page and or data such as pictures, files, etc on to the router system cache so that if another LAN user requests the same data and it had not changed on the website since it was cached the LAN user would receive the cache data saving bandwidth on the WAN. My question is where exactly are these files stored. I can find the Squid cache directories (and there are many) however I cannot recognize the files. Would I not find HTML, .ZIP, .JPG etc files in the cache if I looked into the directory. What am I missing?

So would I use an additional NIC setup as DMZ if I wanted to monitor activity on the WAN outside of my router for say attempts to access my router from the WAN side?

True. Do you find in any of your installs the need to defrag your router? And if so is there a utility build in with Smoothwall linux or pfSense free bsd? Would there be a utility or way of finding out if a drive is fragmented and causing a performance loss with either of the above systems? Although I have poked around very briefly with linux, and have no experience with free BSD and I started on the internet when it was strictly a unix command line I am a noob with these OS's. Another question, do you find much or any need to run these systems with more than 2 NIC's?

Well would a raptor drive not increase case temps vs a SDD or even a low RPM drive especially in a small form factor case like a mini ITX? A few tech notes for you. After just reinstalling a fresh copy of the latest pfSense V2.0b3 06/25/10 which the ISO is around 98MB the image resulted in a 913MB file and the other difference between Smoothwall and pfSense is that SW created 4 partitions while PF created only one. Most of the packages for PF have failed me, except a few (the www proxy for 1). Some packages failed to automatically uninstall when requested. Would you happen to know how I could find out the power draw on say the 3GHZ ATX machine I am using now vs the MINI ITX I am considering? I would like to use something that requires minimal power as it will be on all the time.

I would recommend doing a test install of pfSense using the snapshots, I am using this now and it seems stable. I have played with the package system which is a nice way to install mods in an automated fashion via the WWW GUI however, some of the mods I would like to install namely Squid and the Squid URL blocker have failed. I think the interface is allot more tunable out of the box. My basic issue now is I am unsure how to FTP into the box, the SSH via Putty is not or does not seem like I can run command lines however, command lines can be sent via the WWW GUI. I would like to be able to block url's only to specific users/IP's or MAC ID's and be able to monitor IM's as well. Nevertheless I am impressed with the interface. The automated mod installation system is similar to say Drupal mods vs. having to hard code everything manually. Here is a link to pfSense snapshots http://snapshots.pfsense.org/

I think if not running a virtual machine which would be wicked easy to restore similar to restoring an image but a whole lot faster, the best way would be just an image of the HDD as this would have the setups, and mod already installed. Vs. a fresh install and then installing the mods and then restoring the settings. The image restore would be pretty fast as there is not much data there anyway. I noticed though that Smoothwall setup 4 partitions on the drive and I am not sure if a restore of all partitions would be needed or just say 1. I am also considering (even though I have switched to pfSense running a mini itx with an aton processor and also using a solid state drive vs. platter. I don't know if the increased speed would even matter though.

Well, I had a limited experiance with Smoothwall and even though it worked fine I setup a pfSense snapshot and I was impressed by the configurability out of the box. I think I will go pfSense on my new router machine project.

Any recommendations for backing up a Smoothwall system. I am currently imaging the drive and its 4 partitions with Drive Snapshot. Any issues I should be aware of doing backups this way? Another question... Would it be possible to FTP into Smoothwall and simply copy all files and directories to another system and would it then be possible to restore the system by copying them all back?

So, the ATOM as described above is far beyond both CPU and memory requirements for an active LAN with at least 4 desktops running. And would you also say that the 2GB RAM as in Darrens smoothwall box is far too much? I am testing a 3GHz with 1GBram P4 system and so far I am only using 6% of my ram, so putting another 1GB would be a waste right? With Smoothwall as the example, what would you say is the most CPU/Memory consuming action or proceedure which if was being used would make a person opt for more memory or power? I ask these questions as if everything continues to go well for this project I would like to purchase a mini itx system for the final router setup and want to make sure to purchase the right setup. Thank you for your help.

What would be the maximum CPU power for Smoothwall or pfSense? When does it become a waste? For example a 3ghz box way to much or just right?

I currently run a Netgear FVS338 and need the following features: Block all MAC ID's except those on white-list Block URL Bandwidth shaping for certain IP's Block services on some IP's and not others And I have seen in Smoothwall the ability to monitor the network live including a log of URL's and pages visited and IM sniffing. I am sure this question has been asked MANY times before but I will do it again. Under these circumstances what would you guys choose to perform these features Smoothwall or pfSense? I have been working on several systems and it seems it always comes back to these two distros. Also taking into consideration the ability to tweak later and/or add more powerful features or monitoring, preferably in a WWW GUI vs. command line if possible. Thanks for the opinions!

So if I understand this right you are crating images of the entire drive? I usually have multiple backups one being a drive image after installation of OS, software and updates and misc settings. All data, documents etc are stored on a separate drive which I then use WinZIP to go into whatever folder I want and grab all files packaging into a zip file (unencrypted) and placed on another drive and also a thumb drive. I use a batch script to perform the actions at a button push. If I need a file I can open the associated zip and extract the file or if I suspect a viral or other bad situation on the OS I can restore the image without effecting the data. In the past I used backup programs in the days of tape drives and/or floppy's until one day when I had to fall back to a backup and not until that moment did I find out my backup software was crap and I lost everything. One error lost me everything in a business environment too boot. For 10+ years I have not had a data loss with WinZIP.

The system I used has 2 NIC's one a 10/100 the other a gb. I had set up the system like this. Netgear gb NIC (green) the LAN side (this would have been the DHCP side if I remember correct) Built in Asustek NIC (RED) the WAN side Cable modem ISP. Smoothwall did probe and see the NIC's without issue and in the beginning I was able to plug in and go without issue. I had set it to provide IP's on the 172 range which it did. I was able to get onto the net with multiple computers and log in to the WWW GUI and FTPS & SSH. Then prior to the issue my system CPU fan went on high, the system did not appear to be frozen and it showed the login on the monitor attached to the smoothwall box. I was suddenly unable to access the web and WWW GUI. I checked my WIN7 box and noticed it now had an IP in the 169 range. I rebooted everything and the Smoothwall box started and went into the logon prompt, the WIN7 box had another 169 IP but was unable to get onto the web or WWW GUI. Due to issues I had with other platforms such as Astaro & pfSense I am feeling as if there is something on the router box the linux does not like. I am tempted to start it again fresh as I don't want to give up but I am not sure it would be worth the aggravation. As before, I am a smoothwall noob without question and have no real linux experience.

Well despite all the help I have hit the rocks again with smoothwall. I am not sure if it is my PC Intel P4 3GH 1GBRam 80GBHDD, but after a fresh Smoothwall install which seemed to work ok (setup using LAN IP's in the 172 range the system suddenly crashed. After a reboot the system was providing the LAN's an IP in the 169 range however, no web GUI or internet access could be had. Although I was excited to setup a mini ITX with smoothwall I have gone back to the Netgear FVS338. As an aside I also tried other distros which failed like pfsense (failed to install but ran as a live CD). Unfortunately too much aggravation. Nuts.

Another thing. Even if I build a Smoothwall to match what was on the show it would be cheaper to add 1GB ethernet that what it would be if I upgraded to a matching router with a new one that would have the 1GB which I would imagine would be around $500+/-.

Yes, I was hoping at a minimum that Smoothwall would have the same features as the FVS338 out of the box and available via GUI. I am getting the feeling that in order to match the netgear I will have to learn and dig into settings via command line. I am hoping that there are plug in's available to match the netgear features however, I am also wondering if there would be any issues similar to downloading a pirated OS. If the software is somehow infected I might never know that information is being copied or rerouted. Anyone else feel this way?

Since I am a Smoothwall noob I may not be aware of settings that will allow me to do what my FVS338 does via its GUI, for example. With my FVS338 I am able to: Block or allow ports for a particular LAN IP * I have only been able to via Smoothwall GUI allow a port for ALL LAN ip's. Limit bandwidth for a particular LAN IP Block URL's and limit this block to an IP or IP list, all others would be allowed to the URL Block ALL MAC ID's and only allow those in whitelist to access LAN or WAN Note the FVS338 is not a wireless router, I have used this for years without issue however, I would like to have more information and control over the network and that's why I tried smoothwall. I have also tired several other open source firewalls such as: IPCop - Did not seem to provide comparable features to my FVS338 pfSense - Could not get it to install on my machine unTangle - Again seemed limited Astaro - Could not install on my machine crashed at software check I assume at this point that Smoothwall can be setup preferably via GUI or command line to perform like my netgear, although I really don't want to have do get into the command line all the time. Since I saw the last HAK5 show it lit a fire for me to start this project, if I can get it setup I will purchase the components similar to the show.

I was wondering if anyone could shine any light on any security concerns I should be prepared for using Smoothwall. I am playing with setting this up to replace my Netgear FVS338 wired router, however it seems that Smoothwall cannot do as much as the netgear can out of the box. Should I be aware of any security issues with Smoothwall out of the box?

After watching the latest episode on how to build a router and Smoothwall I gave it a try and although Smoothwall appeared to work well it failed on my list of minimum wants. I was not able to (or could not figure out) how to allow/block a port for each static IP address assigned. I want to allow some IP's access to http/pop3 etc and not others using 2 NIC's. Does anyone have any ideas as to alternatives to try so that I can build my own router with at the least functions found on a Netgear FVS338? I would love to setup this project and did like the options of live network status and the ability to monitor IM's etc...