Lesson 11

Masks


I've dabbled a bit in watercolors, and one of the techniques a painter can use is to apply masking tape around the intended work area on the canvas. When the painting is finished, the masking tape is removed, leaving a nice, crisp, clean border. There is also a liquid equivalent to masking tape called masking fluid that can be painted onto the canvas with a brush. Like masking tape, this protects the white of the canvas. Stencils are used similarly to paint designs, and the advantage of a stencil is that one stencil can be used to paint the exact same shape over and over.

Photoshop has versions of all these methods. They are called "masks" and function just like masking tape, masking fluid, or stencils. What a mask does is cover a portion of a layer to prevent any "paint" from contaminating that area. The advantage that Photoshop masks have over more physical methods is that they can be feathered into many shades of gray, allowing some paint through, depending on the desired effect.

I will start the explanation with this picture of Kinkakuji in Kyoto (left). Note that the sky is very washed out, nearly white. I'd like to darken the color of the sky to make it more compelling.

Previously, in Lesson 5, I demonstrated how to use selection tools to select one area of the photo to change its color. That is, indeed, a handy function. However, after I've adjusted part of the picture, what if I decide later that I'm no longer satisfied with it and want to change it again? I've done a lot of other editing since then, and that area is no longer selected. I would have to go through the whole selection process all over again, and then I might not get the selection exactly the same, which would leave imperfections in the picture.

The solution to that problem is to turn the selection area into a mask, so that no other parts of the picture are affected, and give the mask its own layer, separate from the rest of the picture. That way I can come back to it and fiddle with that one area at any time.

First, use the selection tools to select the desired area of the picture--in this case, the sky. Next, with that area still selected, create a new layer and click the "Add layer mask" icon at the bottom of the Layers palette. When you do, your palette should look like the picture on the right. Your newly created layer now has the equivalent of masking tape framing the sky. No matter what you paint or fill the layer with, only the part over the sky will show.

Now take a closer look at the mask itself. Alt + click on the mask thumbnail in the Layers palette. Your image should change to look something like the picture on the left. If you look at your foreground and background colors, they will also have changed. By default, the foreground is white and the background is black. Masks are always in grayscale.

The black represents the "masking tape" or hidden area. The white represents the part that can show through to the layer beneath. Gray represents an area where a little shows through, but not at full strength. In my picture, the outline where the sky makes contact with the trees is blurry because I feathered the selection by two pixels. You can paint on the mask itself, if you like. This alters what shows through to the layer beneath. For example, if you paint a big white heart in the black area, then the contents of the masked layer will show through in the shape of a heart. If you paint a big black heart in the white area, the original picture will remain unchanged in the shape of the heart. Try it and see what happens after the next step--you can always use the History palette to go back and undo the change.

After drawing the two hearts, click on the thumbnail of the layer linked to the mask. It is currently completely transparent. As an experiment, fill it with a solid color. First, choose a shade of blue that you'd like for the sky, though you can always go back and adjust the color later, so you don't have to be too precise. Next, select the Paint Bucket tool from the toolbar (right). This tool fills an area similar to the way Edit → Fill works, except that it is dependent upon the color of the area to be filled as well. It will only fill the area that is similar in color to the point that was clicked, rather like the Magic Wand tool. This layer is completely transparent, however, so it fills the entire layer.

Click on the image with the Paint Bucket tool. Note the blue heart where I drew in white over the black part of the mask, and the white (the color of the original sky) heart where I drew in black over the white part of the sky (left). This illustrates how changing the mask affects what "paint" gets on the canvas.

Use the History palette to undo the hearts and then re-fill the sky layer with blue. It should now look like your picture has a nice, bright blue sky. The problem is...it's rather boring. There's no variation in color. The sky can, in reality, be clear and blue without a cloud in sight, but the picture should be an "ideal" version of the sky, and that, to my mind, includes clouds.

You could paint a cloud or two on the sky if you want, but it can be difficult to paint realistic clouds. This is where the filters come into play. With your chosen shade of blue in the foreground and white in the background, click Filter → Render → Clouds. If you're not happy with it as-is, you could repeat the filter to re-randomize the clouds. Holding down the Alt key while clicking on the Clouds filter gives a more intense mix. The result should look something like the picture (right). Another way to make the color more intense is to apply the Difference Clouds filter repeatedly.

If you want to give the clouds a more puffy, billowy appearance, you can further apply Unsharp Mask/Wind/Gaussian Blur as shown in Lesson 10...or you could devise your own filter combination. The one in the picture on the left used Unsharp Mask/Diffuse (anisotropic).

I'm finally satisfied with the sky, but how about the rest of the picture? I'd like to brighten the green of the trees to go along with the "bright sunny day" image I'm trying to create. I can do this by adding an adjustment layer. Also, I want to do the two foreground trees separately from the background trees, so first I'll select just the small island in the foreground with the Lasso and copy/paste it to its own layer. Drag the layer in the Layers palette so that it's on top of the sky. (Don't forget to name your layers so that you know what they are.)

The next step is to select the background trees to make a mask for them. This time, I'm going to select them using the Quick Mask function. This function is equivalent to the masking fluid that a painter might use, because you use the Brush tool to apply it.

Directly below the two squares for the foreground and background colors in the toolbar are two icons that both look like a circle inside a rectangle (right). The one on the right will take you to Quick Mask mode. Make sure that the background is the active layer, choose the Brush tool, and then click on the Quick Mask icon.

The default is for the area you paint to be the area that is hidden, just as with masking fluid. In order to invert this, double click the Quick Mask icon to bring up the dialogue box for its options (left). As the default, no matter what color you originally had in the foreground, your brush will paint red. Here is where you can change the color. You can also set it so that the painted area becomes selected rather than masked. Set the options the way you like and click "OK" or press Enter.

Use the Brush to paint over the trees. You can correct mistakes with the Eraser if necessary. Whatever area you paint will be the area upon which the mask is based--either masked or selected, depending on the options you set in the dialogue box. You can be a bit sloppy when it comes to the border with the sky, because I'll show you how to fix that in a minute. You can also be sloppy over the trees in the foreground, because you saved them to their own layer earlier. Where you need to be careful is along the border with the pavilion.

When you're finished, your image should look something like the picture (right). Click the "Edit in standard mode" icon to the left of the Quick Mask icon.

Once you're back in standard mode, your screen should look like the picture on the left. (If you find that the selection area is inverted, you can always change this by choosing Select → Inverse.) Now we want to adjust it so that it doesn't overlap the sky. With the background layer still active, hold down Ctrl + Alt and click on the thumbnail of the sky mask. This will subtract the selection area of the sky from your current selection. The selection area should now be hugging the tops of the trees.

If you wish, you can do the same thing to subtract the foreground trees from the selection by pressing Ctrl + Alt and clicking on their thumbnail. This isn't necessary, though, because they are on the top layer and will cover over any changes that you make to the background.

Now click the "Create new fill or adjustment layer" icon at the bottom of the Layers palette. When the menu pops up, select Hue/Saturation. Make any changes that you wish, and notice that the only thing in the image that changes is the selection area that you just made. Your Layers palette should wind up looking like the picture (left).

When you finish, the trees should be brighter than before, and the rest of the image should have remained the same. I used adjustments of +20/+40/+10 to obtain this picture (right), but you adjust it to suit your taste. You can always come back and adjust it again later. That's the advantage of using an adjustment layer.

The next thing I want to change is the pavilion's reflection in the water. You can see that it's partially reflected, but I want to enhance the reflection to make the image more striking.

Begin by selecting the pavilion. You can do this using the selection tools or Quick Mask mode, however you feel most comfortable. You don't have to be precise, because you you will be copying the selection to a new layer where you can correct it with the Eraser if you need to. You can also leave it relatively sloppy, since it would make perfect sense for the surrounding background trees to be reflected in the water as well. It's up to your discretion.

Once you have made your selection, copy the pavilion from the background layer and paste it to its own layer. (Don't forget to name the new layer, and make sure that it stays beneath the foreground island layer.) Go to Edit → Transform → Flip Vertical to turn the copy upside down. Use the Move tool to position it where you want the reflection. Use the Healing Brush tool to fix the part where the foreground tree covers the left side of the pavilion, since that shouldn't be in the reflection. When you're finished, the image should look something like the picture on the left.

Change the blend mode of the reflection layer to Overlay and reduce the opacity to 75% (or whatever faintness you think best). Use the Blur, Smudge, and Eraser tools to make the edges of the reflection blend in with the water. I also set the Brush tool for an opacity of 10% and painted with white over any areas I wanted to be fainter. When you've blended the outlines with the surrounding water, apply Filter → Distort → Ocean Ripple at a low magnitude (I used 1/1). The result should resemble the picture on the right.

Now I'm ready to adjust the foreground island. Right now it's rather faint, about the same luminosity and saturation as the background. I want to make it stand out in contrast to the background to give the image a greater sense of depth. For this, I'll use the "make the image enhance itself" trick from Lesson 9. In the Layers palette, drag the island layer to the "Create a new layer" icon to copy it. Click on the top layer (the copy) and change the blend mode to Soft Light. This should make the trees stand out.

I don't want the trees to distract from the pavilion, though, because that's supposed to be the center of attention. To reduce their impact, click on the original island layer and apply Filter → Blur → Gaussian Blur. Use your best judgement as to how many pixels to blur the layer; I used 2px to achieve the result in the picture (left).

Upon further inspection, I decided that the foreground trees simply weren't "sunny" enough. With the top (copy) layer active, hold down the Ctrl key and click on the same layer. This is the keyboard shortcut for Select → Load Selection. It displays the selection area of the island as a marquee so that only this area will be affected. The "Load Selection" function is the Photoshop equivalent of a stencil. Not only can you load the selections of current layers, you can save selections as "alpha channels" (to be discussed in a later lesson) so that you can use the exact same selection area again and again.

Click on "Create new fill or adjustment layer" and choose Brightness/Contrast. Increase the brightness untill the trees look sunny enough. I used +25.

The image is almost finished, I just want to add a nice caption. I used the Eyedropper tool on the sunny side of the pavilion to select a nice light yellow color. With a font called Rage Italic, I typed "Kinkakuji"...but it lacked pizzaz. I added a swirly symbol from a dingbat font called PCOrnaments in another layer, yet it still didn't seem interesting enough (right). If you don't have such a dingbat on your computer, you can add a new layer and draw your own swirl with the Brush to achieve the same effect.

With the top text layer active, create a new layer. The new layer should automatically appear as the active layer on top of the text layer. Leaving that layer active, Shift + Ctrl + click on the "Kinkakuji" layer and the dingbat layer. This loads the shape of the text of both layers added together as a selection area on the new layer. Once the area corresponding to the text is selected, click the "Add layer mask" icon. This applies a layer mask to the top layer so that no matter what you do in the layer, it will only affect the text.

I selected a medium brown color from the shady side of the pavilion as my background color to match the pale yellow of the foreground and then filled the new layer with that background color. I played around with filters until I arrived at a combination that I liked. I used Add Noise (20%)/Crystallize (5)/Water Paper (15, 75, 60) to achieve the pattern, then adjusted the brightness by -10 and contrast by +25 to get the result shown (left). This is what the image would look like without the mask. You can enable or disable the mask either by right-clicking on the mask thumbnail and selecting from the pop-up menu or through the Layer menu.

Next I applied a 120° Drop Shadow and Inner Bevel/Chisel Hard of size 3 px to the masked layer.

Compare the difference that the masked pattern layer makes (below).


Without

With

In fact, I was so pleased with the result that I decided to make a frame for the picture using the same effect layer. I started by enlarging the canvas by 20 pixels in both directions. This meant that the pattern mask layer didn't fill the entire canvas any longer, so I used Edit → Transform → Scale to widen the pattern to the same size as the canvas. This stretched the mask along with it, unfortunately misaligning it with the text layers beneath it, so I had to edit the mask directly.

I opened the pattern mask for editing with Alt + click and filled the whole mask with black. I fixed the text so that it would match the text layers by once again using Shift + Ctrl + click to load the text selections and used Edit → Fill to fill them with white. Then I selected the entire canvas and used Select → Transform Selection followed by Select → Inverse to select a border around the image. I filled the border with white as in the picture (left) to allow the pattern layer to show through as a frame.

When I looked at the result, I didn't like the way the frame cast a shadow over the image (right). (Of course, actual physical frames do cast shadows on their pictures, so if you like the shadow effect, it's okay to keep it.) Thus I deleted Drop Shadow from the style. Instead, I decided to make the letters glow. Since they are already painted a light color, the easiest way to do so without affecting the frame is to apply Gaussian Blur to the two text layers. (The computer will ask you to rasterize the layers first; go ahead and do so.) This blurs their outlines so that they spread out and are visible past the edges of the pattern mask. Using Gaussian Blur on a bottom layer is a common way to make an object appear to glow. To enhance the glow, change the blend mode to Screen.

After that was accomplished, I noticed that with all the new glowiness, I should make the background trees brighter to match. Since they have their own adjustment layer, you can simply click on that layer to bring up its properties. I increased the saturation level from +40 to +60. This makes the foreground island not stand out quite as much, so I changed its blending mode from Soft Light to Hard Light.

That does it for the enhancements to the image! Compare the original picture with the finished product below.


Before

After

Since the Load Selection command is so useful, and the shortcut key combinations so convenient, I will list them again here for reference:

  1. Ctrl + click = Load Selection of clicked layer into active layer
  2. Shift + Ctrl + click = Load Selection of clicked layer and add it to selection already present in active layer
  3. Ctrl + Alt + click = Load Selection of clicked layer and subtract it from selection already present in active layer

Homework

Open an image that could use enhancement. Select areas of the image and create mask layers to adjust properties such as hue/saturation or brightness/contrast. If necessary, change the blending mode or opacity of a newly created layer. If applicable, make an object in the image glow with Gaussian Blur. Finally, add a caption and create a masked pattern layer to spice up the text.

If you're truly ambitious, take the original Kinkakuji image and make it into a night scene!