How to Modularize an Existing Shiny App

Introduction

There are multiple tutorials available online on writing modular Shiny apps. So why one more? Well, when I just started with building modular apps myself, these didn’t do much for me. So I really only learned how to write modules when I had an opportunity to team up with an experienced R Shiny developer. The reason I guess is that Shiny modules is an advanced topic, and you typically get to writing modules only when you finally need to scale your apps – and keep opportunities for further scaling open. This typically means when your app goes into production. By then you probably have already developed multiple apps, and switching over to a way of thinking required to write modules may be challenging. If you don’t know what modules are, I recommend starting here and then coming back to this post. Otherwise, read on.

So, I decided to try a different approach and instead of building a simple modular app from scratch, to go in the opposite direction by breaking down a complex real-life app into modules. Here’s the app’s original non-modular code. Note a single app.R file that contains the entire app. static_assets.R includes some object definitions which I moved to a separate file for convenience. calgary_crime_data_prep.R is not part of the app; it is a data retrieval and cleaning script executed once a month with cron. Running the script each time the app launches would have made it extremely slow and would use way too much bandwidth, as the script downloads and processes 150+ Mb of data on each run.

Why Use Modules, Again?

In short, four main reasons:

1. To simplify scaling your app by adding more functionality. Would be much easier to write one more module than ading even more code to the 675 lines in the non-modular version of app.R while also keeping track of all the different IDs in the same namespace.

2. To re-use the code. This is a big one – modules are functions, and as such they abstract some logic and allow to re-use it multiple times instead of copy-pasting the same code. Note how each module is focused on one specific task.

3. By organizing the code, modules make it easier to reason about the code. Editing, expanding, and debugging a well-structured app is less of a challenge than doing the same with one long, messy sheet of code. Note how much shorter each module is compared to the single app.R file.

4. You can have multiple developers working simultaneously on the same app, each one working on a specific module.

Workflow

So, I already have an R Shiny app. It’s a long sheet of code in the app.R file. How do I break it into modules?

Consider App Structure

First, let’s think about how you’d like to structure your app in the most general sense. For example, if your app uses a navbarPage layout, it can be structured around tab panels, like this:

# Define UI
ui <- fluidPage(

  # UI definition begins
  navbarPage(
    title = "Calgary Crime Data Explorer",
    theme = shinythemes::shinytheme("readable"),
    
    # Map UI
    tabPanel(
      title = "Spatial Analysis",
      icon = icon("map-marked-alt"),
      HTML(html_fix), # load Leaflet legend NA positioning fix
      map_ui("map")
    ),
    # Trend UI 
    tabPanel(
      title = "Trend Analysis",
      icon = icon("chart-line"),
      trend_ui("trend")
    ),
    # About UI
    tabPanel(
      title = "About", 
      icon = icon("question"),
      includeMarkdown("about.md")
    )
  )
  
) # UI definition ends

That’s it, that’s the whole UI definition. Everything else is inside map_ui and trend_ui modules. Same with the server function, which is even shorter:

server <- function(input, output, session) {
  map_server("map")
  trend_server("trend")
} 

But that’s just the top-level structuring. Remember that you can call modules from within modules! This allows you to structure the app pretty much in any way you choose.

Identify Reusable UI Code

To break the “Coder’s Block” (like writer’s block, but for coders), just look for any place in the app where you have copied and pasted the same code. Instead of copy-pasting, abstract this code’s logic as a function. To make things simple, start with shorter code blocks, maybe just repetitive pieces of UI with no server logic in them. For example:

category_ui  <- function(id) {
  
  ns <- shiny::NS(id) # Namespace!
  
  selectInput(inputId = ns("my_category"),
              label = h5("Choose category:"),
              choices = categories,
              selected = categories[1])

}

Now you can use category_ui() wherever you may need to choose a category:

...
category_ui("map"),
...

Moreover, if in the future you decide to create more modules which rely on category selection, you’ll be able to just call category_ui() instead of copying this whole code block.

One very important thing to keep in mind when writing modules, is that modules must be namespaced! Note ns("my_category"). See namespacing below for details.

Write Standalone Server Functions

The next step is to write standalone server-side functions. By standalone I mean these technically are not complete modules that have both UI and server components, even though they are created with shiny::moduleServer().

This task is a bit more complex than abstracting parts of the UI, but not by much. First, identify code sections that perform a specific task and convert them into server functions with shiny::moduleServer(). For example, this function makes an STL decomposition plot:

make_stl_plot <- function(id, dataset, my_area, my_category, plot_title, components) {
  
  moduleServer(id, function(input, output, session) {
    
    filtered_data <- dataset %>% 
      ungroup() %>% # df needs to be ungrouped; else plotly may work incorrectly
      filter(name == my_area, 
             category == my_category) %>% 
      mutate(month = tsibble::yearmonth(month),
             month = as.Date(month))
    
      timetk::plot_stl_diagnostics(.data = filtered_data,
                                   .date_var = month,
                                   .value = count,
                                   .title = plot_title,
                                   .line_color = "#1B9E77",
                                   .message = FALSE,
                                   .feature_set = components) %>% 
      layout(margin = list(t = 80, pad = 5)) %>%
      plotly_build()
    
  })
  
}

The result is a module server function which is called from another module:

...
my_plot <- reactive({
  ...
  plotly_obj <- make_stl_plot("trend", 
                              dataset = crime_dataset(),
                              my_area = input$my_area,
                              my_category = input$my_category,
                              plot_title = plot_title(),
                              components = input$my_components)
  ...
})

Note the syntax you use when building functions with shiny::moduleServer(), which can feel confusing: first goes the function(id, ...) and then shiny::moduleServer() is called from within the main function(id, ...) call. I.e. you create a module not like this:

# WRONG!
make_stl_plot <- moduleServer(id, dataset, my_area, my_category, plot_title, components) {...}

But like this:

make_stl_plot <- function(id, dataset, my_area, my_category, plot_title, components) {

  moduleServer(id, 
               # your custom function:
               function(input, output, session) {
                 ... 
               }
  )
  
}

Note where the server function arguments go: function(id, dataset, my_area, my_category, plot_title, components). moduleServer(id, function(input, output, session) part is standard and should not change. Note the id argument: it is required for namespacing to work and is passed from the main function call to moduleServer(). Other arguments – dataset, my_area, my_category, plot_title, components – are custom arguments and are passed to your custom function (here it is the function that builds an STL plot).

Write Whole Modules

Finally, let’s get to making “proper” modules. These are not too different from server-side functions created with shiny::moduleServer(), but they have both UI and server components. One way of thinking about modules is that these are smaller Shiny apps (although they can’t be run independently without modification). Note that it is also possible to develop an app independently and then add it as a module to another app, provided it takes the same inputs. For this tutorial, names of files containing abstracted UI logic end with _ui.R, files containing functions end with _fn.R, and proper modules end with _mod.R. This naming convention is just for the readers’ convenience.

To decide which part of your app can be turned into a module, remember that a module should have a self-contained UI and server logic, and should reflect your app’s structure. In this example the structure is based on the app’s tabs, so the two proper modules as of the time of writing this are map_mod.R and trend_mod.R.

Bring it All Together

The last and by far the easiest step is to bring it all together in the app.R file, which in case of modular apps usually ends up pretty short. You simply need to do three things:

1. Source modules, functions, and scripts: dir(path = "modules", full.names = TRUE) |> map(~ source(.))

2. Call UI functions in the app’s UI definition, e.g.: map_ui("map").

3. Call server functions in the app’s server definition, e.g.: map_server("map").

Some Ideas for Modular Apps Development

Finally, here are some tips and things to keep in mind when developing modular apps with R Shiny. Some of these I haven’t seen explained elsewhere, and some (e.g. namespacing) I’d like to illustrate with examples.

Namespacing

Remember that modules need to be namespaced:

trend_ui <- function(id) {
  
  ns <- shiny::NS(id)
  
  tagList(
    ...
    category_ui("trend"),
    checkboxInput(inputId = ns("extract_trends"),
                  label = h5("Extract trend with STL decomposition")),
    ...
  )
}

This includes all input and output IDs, e.g.: inputId = ns("extract_trends"), plotlyOutput(ns("my_plotly"), ...), as well as UIs and standalone server functions.

Note however category_ui("trend"). Why no ns() call? Well, that’s because it is already namespaced in category_ui.R. And what about make_stl_plot("trend", ...) function call? It is namespaced at the output ID level: plotlyOutput(ns("my_plotly"), ...). If you follow the reactive graph backwards from the my_plotly output object, you will soon get back to make_stl_plot("trend", ...). If this seems confusing, just remember that you must namespace input and output IDs.

Also, what is ns <- shiny::NS(id)? ns(id) is an optional shorthand for shiny::NS(id). If you don’t use the shorthand, you’d have to use the full form NS(id, "name") each time you need to namespace an ID. I.e. instead of ns("extract_trends") you’d need to write NS(id, "extract_trends"). This arguably saves you a bit of typing.

Event Handlers in Modules

Figuring how to use event handlers in modules took me some time, and I wasn’t able to find good examples elsewhere. So this might be the most valuable part of this post.

Let’s start with a simple openReadme() function that opens a markdown file when the user clicks the “Readme” button:

openReadme <- function(input, output, filename) {
  
    observeEvent(input$readme, ignoreInit = TRUE, {
      showModal(
        modalDialog(
          div(
            tags$head(tags$style(".modal-dialog{width:900px}")),
            fluidPage(includeMarkdown(filename))
          ), easyClose = TRUE
        )
      )
    })
  
}

The function contains an observer that executes when input$readme changes. The input comes from actionButton(inputId = ns("readme"), ...). Note that the actionButton() input ID is (and must be) namespaced, while the openReadme() function does not have ns() anywhere in it. Function call openReadme(input, output, "readme_trend.md") also is not expressly namespaced, instead the function takes input and output arguments, which are standard for server functions. filename is a custom argument (name of the file to be opened).

Importantly, unlike other server functions, event handling functions can not be created with moduleServer():

# WRONG!
openReadme <- function(id, filename) {
  
  moduleServer(id, function(input, output, session) {

    observeEvent(input$readme, ignoreInit = TRUE, {
      showModal(
        modalDialog(
          div(
            tags$head(tags$style(".modal-dialog{width:900px}")),
            fluidPage(includeMarkdown(filename)) # "readme_spatial.md" "readme_trend.md"
          ), easyClose = TRUE
        )
      )
    })
    
  })
  
}

Download handlers are built in the same way:

downloadHTML <- function(input, output, app_state) {
  output$download_html <-
    downloadHandler(
      filename = function() { paste0(app_state$title(), ".html") },
      content = function(file) { htmlwidgets::saveWidget(app_state$widget(), file = file) }
    )
}

I am not sure why event handlers can’t be made with moduleServer(). Maybe they deal with namespacing in some non-standard way? If anyone knows why, please tell me in the comments.

Passing Reactives to Nested Modules

Update: following a suggestion in the comments (by Mkranj), I decided to add this short section to focus a bit more on how to pass reactives to nested modules.

With modular apps, you often need to pass reactive values to nested modules, usually from a higher-level (parent) module to an inner (child) module. In that case, you should not pass the value, but rather pass a reactive expression. In practical terms, this means passing the reactive without a parenthesis: my_reactive instead of my_reactive(). Adding parenthesis causes the reactive to evaluate, and evaluation happens in server scope when the server runs. So if you have my_reactive() in the parent module and pass the results of its evaluation to the nested module, the consumer module will receive the static value returned from the evaluation of my_reactive() in the parent module. But we typically need the child module to be able to use reactive inputs. That’s why reactive expressions should be evaluated (ended with a parenthesis) in the final consumer in the nested module. If this sounds a bit abstract, you’ll find a detailed example in the next section.

Managing Complex Apps with Multiple Reactives

Did you wonder what is the app_state argument in downloadHTML <- function(input, output, app_state) {...}? Creating an app state reactive object is a great way to manage complex apps (inspired by this post). Instead of keeping track of multiple reactives, assign them to a list of reactive values, and then you can pass app_state to functions instead of passing each reactive separately:

# Init empty list of reactive values
app_state <- reactiveValues()

# Update app_state when reactives change
observe({
  app_state$title <- my_map_file_title
  app_state$widget <- leaflet_map
  app_state$dataset <- mapping_selection
}) |> bindEvent(c(my_map_file_title(), leaflet_map(), mapping_selection()))

Note that reactives are not evaluated here, and should be passed to consumers as reactive expressions that will be evaluated downstream in the consumer function. To illustrate, uncomment this code block and run the app. Here’s what it will return in the console:

[1] "Reactive expression (not evaluated):"
reactive({
    title_string() %>% 
      str_remove_all(c("<.*?>")) %>% 
      str_trim() %>% 
      str_to_lower() %>% 
      str_replace_all(" ", "_")
}) 

[1] "Evaluated reactive:"
[1] "all_crime_counts_by_community_from_january_2017_to_may_2023"

A reactive object is evaluated when it ends with a parenthesis, e.g.: my_map_file_title(). Remember that a reactive is a function, an object of class closure. If passed to a consumer function, evaluated reactive will be passed as a value, such as character string "all_crime_counts_by_community_from_january_2017_to_may_2023". And since it has already been evaluated once on app launch, the resulting value won’t update when inputs change. In a Shiny app we generally do not want this behavior because we need the consumer function to use reactive inputs. In other words, we need downloadHTML() to download what the app currently shows instead of a static picture of what it was showing when it launched.

Note how in the downloadHTML() consumer function app_state$title() and app_state$widget() are both evaluated (they end with a parenthesis):

downloadHTML <- function(input, output, app_state) {
  output$download_html <-
    downloadHandler(
      filename = function() { paste0(app_state$title(), ".html") },
      content = function(file) { htmlwidgets::saveWidget(app_state$widget(), file = file) }
    )
}

Finally, you may need a way to make sure you correctly pass objects as evaluated reactives vs reactive expressions, or that you pass objects of proper class or type to server functions¹. If you simply rely on Shiny to tell you if you’ve made a mistake, you’ll face rather mysterious error messages. Instead, there’s a neat way to check for these with stopifnot(), as described here.