10.1145/2896377.2901449acmconferencesArticle/Chapter ViewAbstractPublication PagesmetricsConference Proceedingsconference-collections
research-article
Public Access

Unsupervised Clustering Under Temporal Feature Volatility in Network Stack Fingerprinting

Published:14 June 2016Publication History

ABSTRACT

Maintaining and updating signature databases is a tedious task that normally requires a large amount of user effort. The problem becomes harder when features can be distorted by observation noise, which we call volatility. To address this issue, we propose algorithms and models to automatically generate signatures in the presence of noise, with a focus on stack fingerprinting, which is a research area that aims to discover the operating system (OS) of remote hosts using TCP/IP packets. Armed with this framework, we construct a database with 420 network stacks, label the signatures, develop a robust classifier for this database, and fingerprint 66M visible webservers on the Internet.

References

  1. H. J. Abdelnur, R. State, and O. Festor, "Advanced Network Fingerprinting," in Proc. RAID, Sep. 2008, pp. 372--389. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. P. Auffret, "SinFP, Unification of Active and Passive Operating System Fingerprinting," Journal in Computer Virology, vol. 6, no. 3, pp. 197--205, Nov. 2010.Google ScholarGoogle Scholar
  3. T. Beardsley, "Snacktime: A Perl Solution for Remote OS Fingerprinting," Jun. 2003. {Online}. Available: http://www.planb-security.net/wp/snacktime.html.Google ScholarGoogle Scholar
  4. R. Beverly and A. Berger, "Server Siblings: Identifying Shared IPv4/IPv6 Infrastructure via Active Fingerprinting," in Proc. PAM, Mar. 2015, pp. 149--161.Google ScholarGoogle ScholarCross RefCross Ref
  5. J. Caballero, S. Venkataraman, P. Poosankam, M. G. Kang, D. Song, and A. Blum, "FiG: Automatic Fingerprint Generation," in Proc. NDSS, Feb. 2007, pp. 27--42.Google ScholarGoogle Scholar
  6. Y.-C. Chen, Y. Liao, M. Baldi, S.-J. Lee, and L. Qiu, "OS Fingerprinting and Tethering Detection in Mobile Networks," in Proc. ACM IMC, 2014, pp. 173--180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. A. Crenshaw, "OSfuscate," 2008. {Online}. Available: http://www.irongeek.com/i.php?page=security/code.Google ScholarGoogle Scholar
  8. R. Ensafi, D. Fifield, P. Winter, N. Feamster, N. Weaver, and V. Paxson, "Examining How the Great Firewall Discovers Hidden Circumvention Servers," in Proc. ACM IMC, Oct. 2015, pp. 445--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J. Erman, M. Arlitt, and A. Mahanti, "Traffic Classification Using Clustering Algorithms," in Proc. ACM SIGCOMM MineNet, Sep. 2006, pp. 281--286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The WEKA data mining software: An update," SIGKDD Explorations, vol. 11, pp. 10--18, Jul. 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Heidemann, Y. Pradkin, R. Govindan, C. Papadopoulos, G. Bartlett, and J. Bannister, "Census and Survey of the Visible Internet," in Proc. ACM IMC, Oct. 2008, pp. 169--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M. Handley, and H. Tokuda, "Is It Still Possible to Extend TCP?" in Proc. ACM IMC, Nov. 2011, pp. 181--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T. Kohno, A. Broido, and K. C. Claffy, "Remote physical device fingerprinting," IEEE Transactions on Dependable and Secure Computing, vol. 2, no. 2, pp. 93--108, May 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. D. Leonard and D. Loguinov, "Demystifying Service Discovery: Implementing an Internet-Wide Scanner," in Proc. ACM IMC, Nov. 2010, pp. 109--122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. M. Luckie, R. Beverly, T. Wu, and M. Allman, "Resilience of Deployed TCP to Blind Attacks," in Proc. ACM IMC, Oct. 2015, pp. 13--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. G. S. Manku, A. Jain, and A. D. Sarma, "Detecting Near Duplicates for Web Crawling," in Proc. WWW, May 2007, pp. 141--149. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. C. McNab, Network Security Assessment: Know Your Network.\hskip 1em plus 0.5em minus 0.4em\relax O'Reilly Media, Inc., 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. P. Medeiros, A. Brito, and P. M. Pires, "An Effective TCP/IP Fingerprinting Technique Based on Strange Attractors Classification," in Proc. Data Privacy Management and Autonomous Spontaneus Security, Sep. 2009, pp. 208--221. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. NetApplications, "Market Share Statistics for Internet Technologies." {Online}. Available: http://netmarketshare.com/.Google ScholarGoogle Scholar
  20. Netcraft Web Server Survey. {Online}. Available: http://news.netcraft.com/.Google ScholarGoogle Scholar
  21. Nmap. {Online}. Available: http://nmap.org/.Google ScholarGoogle Scholar
  22. G. Prigent, F. Vichot, and F. Harrouet, "IpMorph: fingerprinting spoofing unification," Journal in Computer Virology, vol. 6, no. 4, pp. 329--342, Nov. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. D. Richardson, S. Gribble, and T. Kohno, "The Limits of Automatic OS Fingerprint Generation," in Proc. ACM AISec, Oct 2010, pp. 24--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. G. Roualland and J.-M. Saffroy, "IP Personality." {Online}. Available: http://ippersonality.sourceforge.net/.Google ScholarGoogle Scholar
  25. Z. Shamsi, A. Nandwani, D. Leonard, and D. Loguinov, "Hershel: Single-Packet OS Fingerprinting," in Proc. ACM SIGMETRICS, Jun. 2014, pp. 195--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. B. Skaggs, B. Blackburn, G. Manes, and S. Shenoi, "Network Vulnerability Analysis," in Proc. IEEE MWSCAS, Aug. 2002, pp. 493--495.Google ScholarGoogle Scholar
  27. M. Smart, G. R. Malan, and F. Jahanian, "Defeating TCP/IP Stack Fingerprinting," in Proc. USENIX Security, Jun. 2000, pp. 229--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. G. Taleck, "SYNSCAN: Towards Complete TCP/IP Fingerprinting," CanSecWest, Apr. 2004.Google ScholarGoogle Scholar
  29. THC-RUT Fingerprint Database. {Online}. Available: https://www.thc.org/thc-rut/thcrut-os-fingerprints.Google ScholarGoogle Scholar
  30. F. Veysset, O. Courtay, O. Heen, and I. R. Team, "New Tool and Technique for Remote Operating System Fingerprinting," Apr. 2002. {Online}. Available: http://www.ouah.org/ring-full-paper.pdf.Google ScholarGoogle Scholar
  31. K. Wang, "Frustrating OS Fingerprinting with Morph," 2004. {Online}. Available: http://hackerpoetry.com/images/defcon-12/dc-12-presentations/Wang/dc-12-wang.pdf.Google ScholarGoogle Scholar
  32. F. V. Yarochkin, O. Arkin, M. Kydyraliev, S.-Y. Dai, Y. Huang, and S.-Y. Kuo, "Xprobe2Google ScholarGoogle Scholar
  33. : Low Volume Remote Network Information Gathering Tool," in Proc. IEEE/IFIP DSN, Jun. 2009, pp. 205--210.Google ScholarGoogle Scholar
  34. M. Zalewski, "p0f v3: Passive Fingerprinter," 2012. {Online}. Available: http://lcamtuf.coredump.cx/p0f3.Google ScholarGoogle Scholar
  35. S. Zander, T. Nguyen, and G. Armitage, "Automated Traffic Classification and Application Identification Using Machine Learning," in Proc. IEEE LCN, Nov. 2005, pp. 250--257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. X. Zhang, J. Knockel, and J. Crandall, "Original SYN: Finding Machines Hidden Behind Firewalls," in Proc. IEEE INFOCOM, Apr. 2015, pp. 720--728.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Unsupervised Clustering Under Temporal Feature Volatility in Network Stack Fingerprinting

                    Comments

                    Login options

                    Check if you have access through your login credentials or your institution to get full access on this article.

                    Sign in
                    • Published in

                      cover image ACM Conferences
                      SIGMETRICS '16: Proceedings of the 2016 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Science
                      June 2016
                      434 pages
                      ISBN:9781450342667
                      DOI:10.1145/2896377

                      Copyright © 2016 ACM

                      Publisher

                      Association for Computing Machinery

                      New York, NY, United States

                      Publication History

                      • Published: 14 June 2016

                      Permissions

                      Request permissions about this article.

                      Request Permissions

                      Check for updates

                      Qualifiers

                      • research-article

                      Acceptance Rates

                      Overall Acceptance Rate 256 of 1,704 submissions, 15%

                    PDF Format

                    View or Download as a PDF file.

                    PDF

                    eReader

                    View online with eReader.

                    eReader
                    About Cookies On This Site

                    We use cookies to ensure that we give you the best experience on our website.

                    Learn more

                    Got it!