The AtmelICE calls itself a USB HID device for better or (mostly) worse. This causes heartburn on MacOS X because the OS claims it and you can't get at it via libusb, so avrdude doesn't work. In the past, the workaround has been a dummy kext that keeps the OS from binding it to the HID stack. High Sierra broke that because it now insists on signed kexts and all of the workarounds to date have dried up.
So I've ponied up for a year of the Apple developer program and asked Apple nicely to allow me KEXT signing privileges with my developer ID. And thus, behold! Here attached is a properly signed dummy kext that you can use to let avrdude see your AtmelICE with High Sierra.
Unpack the zip and move the file to /Library/Extensions, then do a "kextload /Library/Extensions/AtmelICE.kext". You'll get a complaint that some bozo named Nicholas Sayer wants you to approve a kernel extension. Go into system preferences, in the "Security & Privacy" section under "General" you'll see where you can approve that. You can then repeat the kextload command and avrdude should work after that (you may need to disconnect and reconnect your AtmelICE).