This explains how I consolidated & de-duped about 45,000 pictures andmovies from four iPhoto libraries on two different machines and variousdirectories full of images on USB drives into one 33,000 file library.

We had about 25,000 photos in one library from a previous (unsuccessful)attempt at consolidating libraries. We had two further librariescontaining about 13,000 and 10,000 images each, and a fourth iPhotolibrary on a separate machine, plus the aforementioned random grab-bagof directories full of images. (I didn't count how many these were, butthey numbered in the thousands.) After consolidation we ended up with33,000 images and movies in one iPhoto library.

iPhoto has a menu item for finding duplicates, but I was not been ableto make it work; it would just hang trying to find duplicates in one ofthe smaller libraries. And even if I could get that to work I didn'thave the space to copy all the images into one giant library and letiPhoto sort it out. So I wrote a program to help me.

One problem was that the same picture would exist with different filenames, probably due to being imported from devices into multipledifferent iPhoto libraries or to being exported & then imported again inthe aforementioned failed consolidation attempt. We also had differentimages existing with the same name, due to the counter on the cameralooping around. This was OK since iPhoto stores images from differentimports in different directories, but it meant that we couldn't justcopy all the files into one directory and use the names to de-duplicatethem.

Jul 01, 2017 1) Launch Photos, located in your /Applications folder. 2) Open a Finder window and browse to the Photos Library folder located at /Pictures/Photos Libraries. 3) Within the Photos Libraries folder, right-click (or control-click) the image library you wish to merge, then select Show Package Content from the dropdown menu. May 09, 2019 Once the photos have all uploaded, go back to Step 1 with your next Photos library. When you’re done, the last Photos library becomes the one you’ll keep, and you can delete the others. Needless to say, make sure you have good backups first! Merge with PowerPhotos. The $30 PowerPhotos from Fat Cat Software provides a variety of extra.

I wrote a simple program to iterate over all image & movie files in adirectory and create a copy of each named after theMD5 hash of its content. Thusfiles whose content were identical, but had different names, would nowend up with the same name. I did not want to end up with tens ofthousands of files in one directory, so I made my program put each filein a directory named after the 2 first characters of each file's name.Here it is:

Since I didn't have enough disk space to duplicate all the images, Iused hardlinks. This essentially means you just create a new filenamepointing to the same data on disk, without taking up extra data. I alsomade the program create a log file containing the location of each fileand its MD5 hash so I could find out how many files were processed andhow many duplicates were found.

A problem with the MD5 hash approach would be if duplicate copies ofimages had different image metadata. This could be the case is somecopies had had their dates adjusted or if iPhoto where changing the EXIFinformation when you star photos or tag faces in them. Fortunately, alittle experimentation showed that iPhoto does not change the imagefiles but stores this extra information in a separate database. Myapproach would thus mean that we would lose stars & face tags, but weconsidered this an acceptable trade off. Consolidating and de-duping thelibraries would be worth it.

What if the EXIF information had been edited? (Some image managersdo.) Would all hope be lost? No, it would not. The Imagemagick suite oftools comes with the identify program, which allows you to get a hashof just the image data. This would allow us to find more duplicates,even if the EXIF information had been edited; however it runs much,much slower than just using MD5 so I chose not to use it.

One worry I had was how to make sure I got hold of all the originalsfrom iPhoto. Due to the aforementioned space constraint I couldn't justexport all the photos and thus create new copies. However, this turnedout not to be an issue. iPhoto stores its originals in a Mastersdirectory, along parallel directories for Previews and Thumbnails.Thus I just ran my program on the Masters directory. One risk would belosing edits to images, but I was OK with this.

In the end I had 255 directories with on average 130 files in each. Iimported these directories into a brand new iPhoto library roughly tenat a time. Since iPhoto doesn't use hardlinks but copies the photos intoits library (to avoid nasty surprises if someone decided to edit thefile in a different location, no doubt) this ate up disk fast; and I hadto delete the existing libraries under the different accounts to makeroom. Since I had Time Machine backups (yes, plural—I rotate two TimeMachine disks) I felt OK doing this.

We are really happy with the result :-)

Update 2014/05/08: A few days ago I received a question to this postfrom Jeff Ruth. Because it, and its answer, may be useful to more peopleI asked if I could reproduce it here.

Hello and thanks for the post about your iPhoto consolidation andde-duping. I'd love to do something similar. We have three Applelaptops (two MacBooks from 2008 and 2009) and a new Macbook ProRetina. In all, about 3-5 iPHoto libraries, I think. Your projectseems to be what we'd like to do. However, I'm not sure how to do it.If you have a minute, could you please tell me how I can use yourcode, or do this a different way, without the program? I am not aprogrammer, but have used Terminal now and then and am not afraid toexperiment if necessary. Thanks a lot, in advance.

Jeff Ruth

My response:

Do your libraries contain duplicates? If not, I wouldn't bother withthis approach. We only needed to because we had imported the samephotos into multiple photo albums.

The script was intended for developers, but if you want to try youhave to copy the file (dedupe-media.sh) to your local disk, then makeit executable. In Terminal, you do that with ‘chmod +xdedupe-media.sh'. (Without the quotes.) You then run the script likeso:

/path/to/dedupe-media.sh /path/to/iPhotoLibrary/Masters /destination/path

If you want to run this on multiple laptops you may want to make thedestination a single USB disk (or something) so probably/Volumes/My_USBDiskName. This disk will accumulate unique photos.The product is just a big folder of files, which will have to importedinto a new library. All metadata (faces, etc) will have to attachedagain.

Please note this script comes with NO WARRANTY OF ANY KIND. You musttake adequate backup of your libraries before running the above.Although I have made my best attempt at making the script work (and itdoes for me!), I take no responsibility for any data loss you mayexperience.

Update 2015/08/16: One annoyance under iTunes is that the 'rolls' or'albums' feature is utterly meaningless, because you've got 255 albumsnamed '00', '01', '..', 'fe', 'ff'. However, you can fix that by runningexiftool over the resulting files to have them grouped by year andmonth (there aremoreexamples in the exiftool documentation):

This moves the images found in DIR_OF_FILES into a directory structurebased on the year and month the data was taken, as taken from the Exifdata in the image itself. The structure will be like so:

With cameras so readily available, it’s easy to accumulate more photos than you know what to do with. You probably take dozens of pictures when you go out, meet up with friends, or go on vacation. And while taking photos is fun, organizing them is work. But it doesn't have to be that way. In this article, we’ll show you how to organize pictures on your Mac with the Photos app and without it. Here are the things you’ll be reading about:

• How to merge your photo libraries in the Photos app
• How to group pictures using Smart Albums in Photos
• Tricks for keeping your Photos library organized
• How to organize pictures outside of the Photos app

Tips to help organize photos on a Mac

1. Consolidate an all your photos in the Photos app for Mac

When macOS Yosemite was released in 2014, the Photos app finally came to Macs, replacing outdated iPhoto. Now with Photos, you’ve got more organization tools, including the ability to switch between libraries and merge them. To start with, let’s get all of your photos from different places onto your Mac. Here’s what you should do:

1. Use iCloud Photo Library to ensure that everything you take on an iOS device is synced automatically with your Mac. Alternatively, AirDrop photos from your iPhone and/or iPad to your Mac.
2. Import photos from external drives to the Photos app on your Mac
3. Import photos from your camera
4. If you have several libraries in the Photos app, merge them into one

Once everything is on your Mac and in one library, it’s easier to get organized. Let’s take a look at a few questions most people have when they juggle several Photos libraries.

How to get rid of duplicate Photos?

The next thing is to get rid of the duplicate and similar-looking pictures that are wasting space on your hard drive.Thankfully, the app Gemini 2 can scan your entire hard drive to quickly find and get rid of them. After you’ve downloaded and installed Gemini 2, you can follow these steps to declutter your photos:

1. Download Gemini 2 for free, install and launch the app.
2. Click + > Pictures Folder.
3. Then, click Scan for Duplicates and let it go.
4. Choose either Review Results or Smart Cleanup.

Review Results does exactly what it promises and will let you see your duplicate files grouped together, making it that much easier to decide which version you want to delete. While Smart Cleanup will get rid of your duplicates with just one click.

How do I switch libraries in Photos?

Libraries in Photos can come from a few different sources, including external devices such as cameras and iPhones. If you were using iPhoto prior to Photos, each device and source will have a Systems Photo Library. You can switch between them, and here is how you do that:

1. Go to Launchpad
2. Hold down the Option key, and then double-click on the Photos app to see the libraries you have on this device
3. Now you can move easily from one photo library to another and open the one you want by double-clicking it

How to create a new Photo library on Mac?

Creating a new library is easy. If Photos is open, you’ll need to quit and reopen to create a new library:

1. Hold down the Option key
2. Double-click on the Photos apps
3. Click the Create New button, then choose Library
4. Give your new library a name and click OK

How to merge two or more Photos libraries?

At present, there is no native macOS tool for merging libraries in Photos. Essentially, what you’ll be doing is moving all pics to one new library. So, create one as described above, and then follow these steps:

1. Launch Photos holding down the Option key and select one the libraries you want to merge
2. In the menu bar, click Edit > Select All
3. Again in the menu bar, select File > Export. Now you’ve got the following options:
• Export Unmodified Original. This will export every image in these libraries, excluding any edits or changes made to them.
• Export [number of photos you want to export] Photos, which gives you the option to pick a format and select Full Size. That will export your photos as they are, with any edits you’ve made, in TIFF, PNG, or JPEG formats.
4. Pick where they’re going (this could be a folder on your Mac or a connected external drive, for example)
5. Repeat these steps for the other libraries you want merged
6. Now close Photos and re-open using Option to select the library you are merging these photos into
7. Either use File > Import or drag the exported photos into the library

How do I delete a Photo library from my Mac?

If you want to delete a library, it’s super easy. You just navigate to the Pictures folder on your Mac, locate the library you want gone, and move it to the Trash. However, you’ll want to make sure you’ve got all the images and videos out of it first. Export them like we described in the previous section, and then trash the library.

2. Use Smart Albums in Photos to organize your pics

After merging your libraries and clearing out duplicates and similar images, take photo organizing one step further and group the remaining pics with Smart Albums. Let’s take a look at how to use this fantastic feature of the Photos app to your advantage.

What is a Smart Album in Photos?

Smart Albums are a way of grouping images by the parameters you set. Maybe you’ll want an album to contain photos taken last Christmas, or you’d rather group all the pics of your kid together. All that and more can be done with a Smart Album. As is the case with a regular Photos album, you can edit and delete it.

How to make a new Smart Album in Photos?

Creating a new Smart Album is simple:

1. Launch Photos
2. Click the + button next to My Albums in the sidebar
3. Select Smart Album<
4. Give your Smart Album a name
5. Choose the Parameters, e.g. “kids,” “days out,” “Holiday 2018.” You can modify these at any time.
6. Click OK to create

By developing a system (e.g., photos are grouped into albums by date taken) and using Smart Albums, you’ll make sure your photo library is easy to navigate, even if it contains thousands of photos from 2001 and up until now.

3. Other tricks for keeping your Photos library organized

Now that you’ve got your libraries consolidated and you’re using Smart Folders, here are a couple more tricks to help keep your pictures organized.

How to favorite Photos?

The more pictures you have in your Photos app, the harder it can be to find the picture you’re looking for. A simple way to view only the photos that are important to you is to favorite them. You can favorite an image in the Photos app either by:

• Selecting a picture and clicking the heart button in the toolbar.
• Hover your mouse over an image and click the heart in the bottom left corner.

Then, you can view all the pictures you’ve favorited by clicking Favorites in the left sidebar.

How to find faces on Photos?

Photos also has a facial recognition feature, which is great when you’re looking for pictures of a specific person. But it’s definitely not without its flaws. It will identify faces and make suggestions, but it requires you to put in some time to either verify the people in a picture or label them yourself. To view your pictures organized by faces:

1. Click People in the sidebar.
2. Select one of the people to view all of their photos.

How to create a shared album?

Shared albums are a collaborative way to share photos with your family friends. Everyone can drop their pictures from a specific event or place into one place. Follow these steps to create a Share album:

1. In the left sidebar of Photos, hover over Shared Albums.
2. Click the +.
3. Name your album and invite people to collaborate.
4. Click Create.
5. Click All Photos in the sidebar.
6. Drag and drop the pictures you want to add to that album.

If you forget to add someone when you’re creating the album, don’t worry. You can always add people to an album by clicking the button that looks like a person in a circle in the toolbar.

How to navigate through years and months?

Consolidating Photo Libraries On Facebook

Probably the most common way to view your photos is by the date they were taken. In the toolbar at the top, you can select to group your pictures by year, month, or even by date.

Even if you’re viewing your pictures in the All Photos tab, you’re still scrolling through them by date. You can use the scale slider to zoom out and see more photos from a time period or zoom in to see a specific period.

Consolidating Photo Libraries On The Internet

4. How to organize photos outside the Photos app

Of course, you don’t need the Photos app to organize your pictures. You can manage them within folders as you do with your documents or downloads. For some, this way maybe a little more difficult because you’re not able to see all of your pics simultaneously. But this does make it easier to transfer your photos or store them on an external hard drive.

Tip:

When you’re not storing your pictures in the Photos app, it does make it harder to spot duplicates, though. So you should use an app like Gemini 2 and periodically scan for duplicate or similar-looking photos.

The app will scan your hard drive to show you copies of the same picture that are just wasting space. You can get rid of your duplicates with one click of the Smart Cleanup button or you can review the files Gemini 2 finds to decide which copies you want to delete. And the best part is you can download Gemini 2 and try it for free!

How to find all photos on Mac?

Consolidating Photo Libraries On Iphone

If you have pictures everywhere on your computer and you’re wondering how to find the photo files on your Mac, the easiest way is with Smart Folders in Finder. Setting up a Smart Folder will allow you to see all of your pictures in one Finder window, regardless of where they’re actually stored. To set up a Smart Folder for your pictures:

1. In a new Finder window, click File > New Smart Folder.
2. Click “This Mac” and then click the + in the right corner.
3. From the drop-down menus, change Name to Kind and Any to Image.
4. If you want to save the Smart Folder so you can come back to it, click Save and then name the folder.

Consolidating Photo Libraries On Mac

But the best way to organize your photos on a Mac is in a detailed folder structure, typically by date of when you took the pictures. And once you have a Smart Folder where you can find and see all of your photos in one place, it makes it a little bit easier to sort and organize them into easy-to-navigate folders.

How to view pictures on Mac?

Viewing pictures on a Mac without Photos is easy, but can get tedious really quickly. You can use Preview to open and view your pictures. The major drawback to doing this is you first have to find the photo file(s) in Finder and then double-click to open them with Preview.

That’s it, hope this short guide helps you cope with your photo overload. And remember: the best way to organize photos on your Mac is to keep your library lean and manageable — free from clutter and useless copies. Gemini 2 will gladly help with that.

Consolidating Photo Libraries One

These might also interest you: