scale_*_manual drops NA legend value if values argument is named

[beansrowning]

I racked my brain on this one for hours trying to figure out what I was doing wrong. I think I've identified either a bug or undocumented behavior of using a manual scale.

The problem seems to stem from passing a named vector to values. NA values are translated and displayed correctly on the plot, but the label in the legend is dropped.

It seems like there was some discussion of this issue when the behavior of values changed (#4471 (comment), #4619), but it seems like it was resolved (in favor of not displaying an explicit NA value in the legend regardless of whether values is named, or values + labels are passed.

If that's the case, this appears to be a regression. Otherwise, it would be awesome if the docs pointed out how to add an explicit NA level, since this is often required/requested in my space.

library(ggplot2)
#> Warning: package 'ggplot2' was built under R version 4.2.3
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(scales)
library(sf)
#> Linking to GEOS 3.9.1, GDAL 3.4.3, PROJ 7.2.1; sf_use_s2() is TRUE

# Read in NC Example
nc_shp <- system.file("shape/nc.shp", package="sf")

agr <- c(
  AREA = "aggregate", PERIMETER = "aggregate", CNTY_ = "identity",
  CNTY_ID = "identity", NAME = "identity", FIPS = "identity", FIPSNO = "identity",
  CRESS_ID = "identity", BIR74 = "aggregate", SID74 = "aggregate", NWBIR74 = "aggregate",
  BIR79 = "aggregate", SID79 = "aggregate", NWBIR79  = "aggregate"
)

nc <- st_read(nc_shp, agr = agr, quiet = TRUE) |>
  mutate(
    value = runif(100, 0, 100),
    value_bin = cut(value, 4)
  )


# Assigned color brewer scale
col_scale <- levels(nc$value_bin)
col_scale <- setNames(
  brewer_pal("seq")(length(col_scale)),
  col_scale
)

# Randomly remove 10 values
to_na <- sample(seq_len(nrow(nc)), 10)
nc[to_na, "value_bin"] <- NA_character_


# NA displays if color is discrete
ggplot(nc) +
  geom_sf(aes(fill = value_bin)) +
  labs(title = "North Carolina", subtitle = "default scale")

# NA is dropped in manual scale
# if values is a named vector
ggplot(nc) +
  geom_sf(aes(fill = value_bin)) +
    scale_fill_manual(values = col_scale, na.value = "#CCCCCC") +
    labs(title = "North Carolina", subtitle = "manual scale")

# Explicit na.translate fails here, too
ggplot(nc) +
  geom_sf(aes(fill = value_bin)) +
    scale_fill_manual(values = col_scale, na.value = "#CCCCCC", na.translate = TRUE, drop = FALSE) +
    labs(title = "North Carolina", subtitle = "manual scale + na.translate")

# Passing an unnamed vector seems to fix it
ggplot(nc) +
  geom_sf(aes(fill = value_bin)) +
    scale_fill_manual(values = unname(col_scale), na.value = "#CCCCCC") +
    labs(title = "North Carolina", subtitle = "un-named manual scale")

# This impacts other geoms/scales as well
nc |>
  as_tibble() |>
  select(AREA, PERIMETER, value_bin) |>
  ggplot(aes(x = AREA, y = PERIMETER, color = value_bin)) +
    geom_point() +
      scale_color_manual(values = col_scale, na.value = "#CCCC00", na.translate = TRUE) +
      labs(title = "Area x Perimeter, North Carolina Counties", subtitle = "named manual scale")

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23 ucrt)
#>  os       Windows 10 x64 (build 19044)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  English_United States.utf8
#>  ctype    English_United States.utf8
#>  tz       America/New_York
#>  date     2023-04-26
#>  pandoc   2.11.2 @ C:\\Users\\<xxx>\\AppData\\Local\\Programs\\RStudio\\bin\\pandoc/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version date (UTC) lib source
#>  assertthat     0.2.1   2019-03-21 [1] CRAN (R 4.2.2)
#>  class          7.3-20  2022-01-16 [1] CRAN (R 4.2.1)
#>  classInt       0.4-8   2022-09-29 [1] CRAN (R 4.2.1)
#>  cli            3.4.1   2022-09-23 [1] CRAN (R 4.2.1)
#>  colorspace     2.0-3   2022-02-21 [1] CRAN (R 4.2.1)
#>  curl           4.3.3   2022-10-06 [1] CRAN (R 4.2.1)
#>  DBI            1.1.3   2022-06-18 [1] CRAN (R 4.2.1)
#>  digest         0.6.29  2021-12-01 [1] CRAN (R 4.2.1)
#>  dplyr        * 1.0.10  2022-09-01 [1] CRAN (R 4.2.1)
#>  e1071          1.7-11  2022-06-07 [1] CRAN (R 4.2.1)
#>  evaluate       0.17    2022-10-07 [1] CRAN (R 4.2.1)
#>  fansi          1.0.3   2022-03-24 [1] CRAN (R 4.2.1)
#>  farver         2.1.1   2022-07-06 [1] CRAN (R 4.2.1)
#>  fastmap        1.1.0   2021-01-25 [1] CRAN (R 4.2.1)
#>  fs             1.5.2   2021-12-08 [1] CRAN (R 4.2.1)
#>  generics       0.1.3   2022-07-05 [1] CRAN (R 4.2.1)
#>  ggplot2      * 3.4.2   2023-04-03 [1] CRAN (R 4.2.3)
#>  glue           1.6.2   2022-02-24 [1] CRAN (R 4.2.1)
#>  gtable         0.3.1   2022-09-01 [1] CRAN (R 4.2.1)
#>  highr          0.9     2021-04-16 [1] CRAN (R 4.2.1)
#>  htmltools      0.5.3   2022-07-18 [1] CRAN (R 4.2.1)
#>  httr           1.4.4   2022-08-17 [1] CRAN (R 4.2.1)
#>  KernSmooth     2.23-20 2021-05-03 [1] CRAN (R 4.2.1)
#>  knitr          1.40    2022-08-24 [1] CRAN (R 4.2.1)
#>  labeling       0.4.2   2020-10-20 [1] CRAN (R 4.2.0)
#>  lifecycle      1.0.3   2022-10-07 [1] CRAN (R 4.2.1)
#>  magrittr       2.0.3   2022-03-30 [1] CRAN (R 4.2.1)
#>  mime           0.12    2021-09-28 [1] CRAN (R 4.2.0)
#>  munsell        0.5.0   2018-06-12 [1] CRAN (R 4.2.1)
#>  pillar         1.8.1   2022-08-19 [1] CRAN (R 4.2.1)
#>  pkgconfig      2.0.3   2019-09-22 [1] CRAN (R 4.2.1)
#>  proxy          0.4-27  2022-06-09 [1] CRAN (R 4.2.1)
#>  purrr          0.3.5   2022-10-06 [1] CRAN (R 4.2.1)
#>  R.cache        0.16.0  2022-07-21 [1] CRAN (R 4.2.1)
#>  R.methodsS3    1.8.2   2022-06-13 [1] CRAN (R 4.2.0)
#>  R.oo           1.25.0  2022-06-12 [1] CRAN (R 4.2.0)
#>  R.utils        2.12.0  2022-06-28 [1] CRAN (R 4.2.1)
#>  R6             2.5.1   2021-08-19 [1] CRAN (R 4.2.1)
#>  RColorBrewer   1.1-3   2022-04-03 [1] CRAN (R 4.2.0)
#>  Rcpp           1.0.9   2022-07-08 [1] CRAN (R 4.2.1)
#>  reprex         2.0.2   2022-08-17 [1] CRAN (R 4.2.2)
#>  rlang          1.1.0   2023-03-14 [1] CRAN (R 4.2.3)
#>  rmarkdown      2.17    2022-10-07 [1] CRAN (R 4.2.1)
#>  s2             1.1.0   2022-07-18 [1] CRAN (R 4.2.1)
#>  scales       * 1.2.1   2022-08-20 [1] CRAN (R 4.2.1)
#>  sessioninfo    1.2.2   2021-12-06 [1] CRAN (R 4.2.2)
#>  sf           * 1.0-8   2022-07-14 [1] CRAN (R 4.2.1)
#>  stringi        1.7.8   2022-07-11 [1] CRAN (R 4.2.1)
#>  stringr        1.4.1   2022-08-20 [1] CRAN (R 4.2.1)
#>  styler         1.7.0   2022-03-13 [1] CRAN (R 4.2.1)
#>  tibble         3.1.8   2022-07-22 [1] CRAN (R 4.2.1)
#>  tidyselect     1.2.0   2022-10-10 [1] CRAN (R 4.2.1)
#>  units          0.8-0   2022-02-05 [1] CRAN (R 4.2.1)
#>  utf8           1.2.2   2021-07-24 [1] CRAN (R 4.2.1)
#>  vctrs          0.5.1   2022-11-16 [1] CRAN (R 4.2.2)
#>  withr          2.5.0   2022-03-03 [1] CRAN (R 4.2.1)
#>  wk             0.6.0   2022-01-03 [1] CRAN (R 4.2.1)
#>  xfun           0.33    2022-09-12 [1] CRAN (R 4.2.1)
#>  xml2           1.3.3   2021-11-30 [1] CRAN (R 4.2.1)
#>  yaml           2.3.5   2022-02-21 [1] CRAN (R 4.2.1)
#> 
#>  [1] C:/Users/<xxx>/AppData/Local/Programs/R/R-4.2.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[beansrowning]

And now I see this is probably a dup of #5214, sorry. I tried searching to no avail.

[teunbrand]

Indeed the underlying problem seems identical. Thanks for your report with reproducible examples regardless! I'll close this as a duplicate then.