Category: Uncategorized

Bulk Delete Duplicate Files Using czkawkas Results File

czkawka tool to find duplicates works pretty well. I ran it against my NAS as I want to migrate data from that aging device to a new device and figured I would take the time to clean up many duplicate files. I have a previous post from the other day that details some of my journey with it so far and use against NAS devices.

The one thing I can’t figure out in czkawka is if there is an ability to, for each group of duplicates, delete everything but the first file listed in that group. What I’ve had to do is manually put checkmarks on the list. Fine and dandy for maybe a couple hundred duplicates, but I have about 15k duplicates in about 12k groups accounting for about 100GB of wasted space. That’s a LOT of clicking.

czkawka allows me to save a list of the duplicate files found. Here’s a sample structure of that results file.

-------------------------------------------------Files with same hashes-------------------------------------------------Found 14934 duplicated files which in 12706 groups which takes 96.99 GiB.

---- Size 1.76 GiB (1888739918) - 2 files
C:\nas_photography\1\IMG_7467.MOV
C:\nas_photography\1\New folder\DCIM\924BJHKG\IMG_7466.JPG

---- Size 748.34 MiB (784696092) - 2 files
C:\nas_photography\1\IMG_7524.MOV
C:\nas_photography\1\New folder\DCIM\924BJHKG\IMG_7523.JPG

---- Size 688.21 MiB (721643510) - 2 files
C:\nas_photography\1\IMG_7656.MOV
C:\nas_photography\1\New folder\DCIM\924BJHKG\IMG_7655.JPG

Here’s my attempt to “automate” this. I think I put about an hour of work into this (mostly fiddling with WSL trying to get the Windows share mount permissions working — which I didn’t get working).

#!/bin/bash

lineno=0
while IFS= read -r line
do
  if [[ "$line" =~ ^---- ]]; then
    lineno=1
  elif [[ "$line" =~ ^$ ]]; then
    lineno=0
  else
    ((lineno++))
    if [ "$lineno" -gt "2" ]; then
      echo del /q \""$line"\" >> windows_delete_duplicates.bat # For windows
      # echo rm -f \""$line"\" >> linux_delete_duplicates.sh # For linux
    fi
  fi
done < results_duplicates.txt

czkawka – Scan a NAS in Windows

I came across a nice tool named czkawka that “is a simple, fast and free app to remove unnecessary files from your computer.” I want to use this against an old NAS drive with tens of thousands of photos (and files) to find the duplicates and clean them up so I can move the 2TB of data to a different storage solution.

I couldn’t get the application to utilize the smb:// protocol as it suggests, for whatever reason (probably because I’m doing this in Windows). So here’s how I managed to give access to the application for my NAS share.

net use P: \\mediaserver\pictures
mklink /D C:\NAS_Pictures P:\
symbolic link created for C:\NAS_Pictures <<===>> P:\

Now in czkawka, I add the C:\NAS_Pictures source folder and begin to scan…

LetsEncrypt with ServiceDeskPlus

I thought I had typed this up at one point, but apparently I can’t find it on the blog. Here’s the steps I take to install and renew SSL certificates for ServiceDeskPlus MSP using LetsEncrypt. I’ve adapted these notes for public consumption.

My environment is a Windows Server that hosts the SDP MSP instance.

On the SDP MSP host, generate a certificate. I am using Certbot Win32 on the Windows host.

certbot certonly --standalone -d servicedesk.example.com

The generated certificate files will be in: C:\certbot\archive\servicedesk.example.com

Create PFX for import into ServiceDeskPlus MSP

Copy the privkeyX.pem, certX.pem, chainX.pem from the above directory to a machine that has OpenSSL tools installed (most Linux).

NOTE: Where X is the number in iteration; at the time of this documentation it is 2.

Issue the following command to generate a PFX, specifying the passphrase to use.

  • Linux with OpenSSL package installed:
openssl pkcs12 -export -out certificate.pfx -inkey privkey.pem -in cert.pem -certfile chain.pem
  • Windows host with OpenSSL 32/64 installed:
"\openssl\bin\openssl.exe" pkcs12 -export -out certificate.pfx -inkey privkey2.pem -in cert2.pem -certfile chain2.pem -password pass:PASSPHRASEHERE

Prepare the Certificate for Import into SDP MSP

Copy the certificate.pfx file to your computer

Update ServiceDeskPlus MSP

  1. Log into ServiceDeskPlus MSP web. Go to Admin. Search for SSL (it will load the Import SSL Certificate).
  2. Select the generated certificate.pfx on the webpage, providing the passphrase set above.
  3. Once imported successfully, must restart ServiceDeskPlus MSP. Can be done on the Windows server in Services.

Error Importing?

NOTE: The Java version of SDP MSP appears to not support the algorithm used to create the certificate when I try to import via the SDP MSP web admin interface. I get the following error in the log dump:

Invalid KeySpec: unknown PRF algorithm 1.2.840.113549.2.9

So work around is to Import the certificate.pfx that was just created into the computer’s Personal store then export it using TripleDES-SHA1 encryption.

Import:

  1. Double click the certificate.pfx file which launches an import certificate wizard.
  2. Click the Local Machine and click Next.
  3. Check the file location of the certificate and press Next.
  4. Enter the Password and enable “Mark this key as exportable” and press Next.
  5. Choose the appropriate certificate store, in this example the personal certificate store is chosen. After that, press Next.
  6. Click Finish to complete the import.

Export:

  1. Launch Microsoft Management Console. Press Win+R, type in mmc and press OK.
  2. Click File and select the Add/Remove Snap-in option.
  3. Click on Certificates in the list of Available snap-ins and then, on the Add button.
  4. Select Computer account and click Next.
  5. Choose Local Computer and click on the Finish button.
  6. Click OK to add the certificate snap-in and get back to console.
  7. Expand the Personal store in the left-side menu, and choose Certificates. Right-click on the certificate you want to export All Tasks > Export.
  8. This will run the Certificate Export Wizard.
  9. If the radio button ‘Yes, export the private key’ is grayed out, it means that either the private key was not marked as exportable during the certificate request generation, or that you do not have the corresponding private key on the machine you are using.
    • Note: if you used IIS Manager certificate request wizard to generate the CSR code, the private key will be marked as exportable by default. In this case, you will not be able to create a PFX file, only export the certificate without the private key. To have the opportunity to export the certificate to another machine, you will need to create a new CSR code marking the private key as exportable and perform a certificate reissue. Otherwise, you can generate a new CSR code for the same common name on the new machine and import the certificate to it after the reissue is completed.
  10. If you can export the private key, proceed to the next stage. The window Export File Format will have the format Personal Information Exchange – PKCS #12 (.PFX) selected. Please check Include all certificates in the certification path if possible to have the certificate exported with the chain of intermediate CA certificates into a .pfx file. Then click Next.
    • Note: do not choose ‘Delete the private key if the export is successful’.
  11. Type and confirm password on the next window and click Next. Make sure you remember the password; it will be used later during the import of a .pfx file to a new server.
    • Note: The encryption method used for the password has to be ‘TripleDES’. Also, the password should not contain the symbol ‘&’
  12. In the File to Export window select the name and location of the .pfx file to which the certificate and private key will be exported.
  13. Click Finish to complete the export wizard. The certificate has been successfully imported.

Update ServiceDeskPlus MSP:

  1. Log into ServiceDeskPlus MSP web. Go to Admin. Search for SSL (it will load the Import SSL Certificate).
  2. Select the generated certificate.pfx on the webpage, providing the passphrase set above.
  3. Once imported successfully, must restart ServiceDeskPlus MSP. Can be done on the Windows server in Services.

Other Thoughts

I think I can take advantage of certutil on Windows to import/export the certificate to avoid a lot of manual work. I know I can import the certificate easily, just haven’t tinkered with exporting it to meet the requirements of SDP MSP.

Something like this will import: certutil -f -p PASSPHRASE -importpfx certificate.pfx

ChromeOS Flex Download

Direct download of the ChromeOS Flex bin files are available at Chromium Dash.

The ChromeOS Flex recovery does not work on Linux, but you can use the following to workaround that.

Here are the instructions for the Linux script:   

  1. On the Linux computer, download the Recovery Tool.
  2. Change the script permissions to allow execution with the following command: 
    sudo chmod 755 linux_recovery.sh
  3. Run the script with root privileges with the following command: 
    sudo bash linux_recovery.sh --config https://dl.google.com/dl/edgedl/chromeos/recovery/cloudready_recovery.conf
  4. Follow the on-screen instructions to create recovery media.
  5. To continue to recover your Chromebook, follow the steps above.  
--config https://dl.google.com/dl/edgedl/chromeos/recovery/cloudready_recovery.conf

Here are a couple photos of me running this on a Lenovo ThinkCentre M70z 16GB RAM i5-10400U 6 core with 512GB SSD.

Uninstall ManageEngine DesktopCentral Agent

Here are some methods to remove the ManageEngine DesktopCentral Agent.

@echo OFF
Rem Manage Engine Desktopcentral Agent Uninstallation String
Rem This bat file will uninstall the Desktopcentral Agent
%SystemRoot%\system32\msiexec.exe /x{6AD2231F-FF48-4D59-AC26-405AFAE23DB7} MSIRESTARTMANAGERCONTROL=Disable REBOOT="ReallySuppress" /qn

Agent Uninstall Tool

https://www.manageengine.com/products/desktop-central/desktop_agent_uninstallation.html

Local Copy:

AgentCleanupTool

Agent Uninstall VBScript

'Manage Engine Desktopcentral Agent .

'Script to Clean up ManageEngine Desktop Central Agent from Add remove programs .
'================================================================================

On Error Resume Next

Set WshShell = WScript.CreateObject("WScript.Shell")
   WshShell.RegRead("HKEY_CLASSES_ROOT\Installer\Products\F1322DA684FF95D4CA6204A5AF2ED37B\SourceList\Net\")
   WshShell.RegDelete "HKEY_CLASSES_ROOT\Installer\Products\F1322DA684FF95D4CA6204A5AF2ED37B\SourceList\Net\"
   WshShell.RegDelete "HKEY_CLASSES_ROOT\Installer\Products\F1322DA684FF95D4CA6204A5AF2ED37B\SourceList\Media\"
   WshShell.RegDelete "HKEY_CLASSES_ROOT\Installer\Products\F1322DA684FF95D4CA6204A5AF2ED37B\SourceList\"
   WshShell.RegDelete "HKEY_CLASSES_ROOT\Installer\Products\F1322DA684FF95D4CA6204A5AF2ED37B\"

'Uninstall Agent and Remote Control Service
'==========================================
WshShell.Run "%windir%\system32\sc stop "   &Chr(34)& "ManageEngine Desktop Central - Agent"  &Chr(34),1,True
WshShell.Run "%windir%\system32\sc delete " &Chr(34)& "ManageEngine Desktop Central - Agent"  & Chr(34),1,True
WshShell.Run "%windir%\system32\sc stop "   &Chr(34)& "ManageEngine Desktop Central - Remote Control"  &Chr(34),1,True
WshShell.Run "%windir%\system32\sc delete " &Chr(34)& "ManageEngine Desktop Central - Remote Control"  & Chr(34),1,True

'**********************************************************************************************************************