Isaac Bell

3 Approaches to Mex Algorithms

Isaac Bell — Tue, 08 Aug 2023 21:00:54 GMT

Mex problems refer to the minimum non-negative integer that is not present in an array of positive integers. Solving Mex problems requires finding an efficient algorithm to determine the missing number. In this blog post, we’ll explore three different approaches to solving Mex problems.

1. Quick Mex for Small Array Values

The first approach is to use a set when the values in the array are small, specifically when all values are less than 1 million. This approach takes advantage of the set’s constant time lookups. We can create a set and insert each element from the array into the set. Then we loop through the numbers from 0 to 1 million and return the first number that is not present in the set.

int mex(vector<int>& numberArray) {
  set<int> sett;

  fo(i, numberArray.size()) 
    sett.insert(numberArray[i]);
  fo(i, 1000001)
    if(!sett.count(i)) return i;
  return -1;
}

Additionally, this approach can be optimized further with coordinate compression.

2. Sorting Approach

The second approach involves sorting the array and then looping through the sorted array. We start with mexas 1 and increment it whenever we come across a number that is equal to mex. After the loop, mex will be the answer. This approach takes O(N log N)time.

sortall(numberArray);
int mex = 1;
for(int e : numberArray) {
 if (e == mex) {
  mex++;
 }
}

3. More Sophisticated Approach

The third approach is a more sophisticated approach that involves removing elements greater than N from the array, sorting the array, and then analyzing the sequence. This approach takes O(N log N)time. The steps are as follows:

1. Go through the array and remove elements that are greater than N.

2. Sort the array.

3. Traverse the sequence and look at the first number that does not correspond to the position in the array.

For example, if we have an array [0, 3, 5, 7, 2, 4, 1, 10, 19] and N = 9, we would remove the numbers greater than 9, leaving us with [0, 3, 5, 7, 2, 4, 1].

Then, we would sort the array to get [0, 1, 2, 3, 4, 5, 7].

Finally, we would traverse the sorted array and look for the first number that does not correspond to its position. In this case, the first number that doesn’t match its position is 6, so the answer is 6.

Conclusion

There are multiple approaches to solving Mex problems, and each approach has its own advantages and limitations. When deciding which approach to use, consider the size of the array and the required time complexity.

If you have questions or comments, please leave them below. Stay tuned for a continuation on how to perform offline MEX queries.

Beginner’s Guide to Ruby Benchmarking

Isaac Bell — Fri, 27 May 2022 20:58:24 GMT

This is a quick step-by-step guide for when you need to do some performance testing on the fly.

Ruby has a built-in benchmarking module that is simple to use and effective enough to cover your standard use cases. Using one of a few functions, we can quickly isolate non-performant sections of our code.

Benchmark Reporting

The Benchmark.bm method gives a printout of performance metrics for any blocks of code we pass into it. We can separate this into multiple chunks of code to execute, to test different things at once. We will receive multiple reports, one for each use of the report method.

Benchmark.bm do |bench|
bench.report('Performance Test 1') do

execute some code here

end
bench.report('Performance Test 2') do

execute some other code here

end
end

For this tutorial, we’ll only be testing one code block:

Benchmark.bm do |bench|
bench.report('Testing some code') do

execute some code here

end
end

This will give us output similar to the following:

user       system     total       real  
0.544000   0.036000   0.580000    (1.049005)

Each report displays the user CPU time, system CPU time, the combined sum CPU time, and the elapsed real time.

The unit of time is seconds.

So our code ran in about 1.05 seconds overall.

Let’s take this a step further and profile our code more thoroughly.

Code Profiling

We want to get an accurate impression of how long our code will take to run on a small sample size of data. Then we’ll assess how well the code will perform with a larger pool of data.

Benchmark.bm do |bench|
bench.report('Testing some code') do
process_100_records
end
end

Let’s run this benchmark 5 times and record the results.

 user       system     total       real  
Run 1: 0.544000   0.036000   0.580000    (1.049005)  
Run 2: 0.500000   0.080000   0.580000    (1.059239)  
Run 3: 0.520000   0.048000   0.568000    (1.026662)  
Run 4: 0.536000   0.048000   0.584000    (1.066434)  
Run 5: 0.496000   0.080000   0.576000    (1.050087)

We’ll average the results. You may want the discard the minimum and maximum runtimes you receive, especially if they visibly deviate significantly from the mean.

For this tutorial, we’ll keep all 5 results.

Here is a quick script to determine what our expected runtime will be with the larger pool of data. For this example, we have a sample size of 100 records and a full data size of 1M records.

# This code can be run in a terminal - just edit the variables

def avg(array)  
  array.sum(0.0) / array.size  
end  

def seconds_to_hms(sec)  
  "%02d:%02d:%02d" % [sec / 3600, sec / 60 % 60, sec % 60]  
end  

# Fill in these variables with your own results  
realtimes = [1.049005, 1.059239, 1.026662, 1.066434, 1.050087]  
sample_size = 100  
full_data_size = 1000000

average_time = avg(realtimes)

`# predicted runtime for full data size  
expected_runtime = average_time / sample_size * full_data_size  

p seconds_to_hms expected_runtime  

`\=> "02:55:02"

This code will take about 3 hours to run(!)

Whether or not that is acceptable will depend on your use case, but now you have the info you need to make that determination.

I hope this helps. Good luck in the field!

Overriding I/O Streams in C++

Isaac Bell — Thu, 06 Jan 2022 17:03:29 GMT

POV: You have a struct/class which you want to add I/O capability to, but you don’t know how.

C++ lets us override the “>>” and “<<” operators, corresponding to input and output streams, respectively.

Let’s build an example; the Point structure holds two values, x and y. It prints information about itself using the cout function upon its creation.

struct Point {
int x, y;

Point(int _x, int _y): x(_x), y(_y) {
std::cout << "Created point with coordinates: " << *this << "\n";
};
};

int main() {
Point p(123, 456);

return 0;
}

Running this code will break because our output stream does not know how to handle our custom data structure. We can override the << operator to clearly define the behavior we want.

friend ostream& operator<<(ostream& os, const Point& point) {
return os << point.x << ' ' << point.y;
}

Adding this class function will cause cout or any other output streaming operation to print the point’s x and y coordinates, with a space separator.

Note that we declare our override to be a “friend function”. The output stream will be where this function is invoked, rather than from within our class.

— — —

Now let’s say we want to read in the point’s coordinates from an input source, such as arguments read through the CLI. We could do this by directly scanning the individual x and y numbers, but we’d like a more elegant solution.

struct Point {
...
};

int main() {
Point p;
cin >> p;

return 0;
}

This can be achieved by overriding the >> operator. The implementation is nearly the same as what we did to override the << operator. Note once again that this is a friend function; in this case, the function will be called from the istream class.

friend istream& operator>>(istream& is, Point& point) {
return is >> point.x >> ' ' >> point.y;
}

Putting this all together, our final example looks like this:

struct Point {
int x, y;

friend ostream& operator<<(ostream& os, const Point& point) {
return os << point.x << ' ' << point.y;
}

friend istream& operator>>(istream& is, Point& point) {
return is >> point.x >> ' ' >> point.y;
}

Point(int _x, int _y): x(_x), y(_y) {
std::cout << "Created point with coordinates: " << *this << "\n";
};
};

int main() {
Point p;
std::cin >> p;
std::cout << "Point coordinates: " << p;

return 0;
}

That’s it, you’re all done. Happy coding!

Doom Emacs Cheatsheet

Isaac Bell — Sat, 30 Oct 2021 20:28:28 GMT

I wanted to share the cheatsheet I wrote for Doom Emacs commands. I wrote this one because the others I found online were out of date. I use and maintain this reference daily.

This is something of a living document, so the document can and will change over time. This command list is not exhaustive, but it does cover all the commands I use regularly.

I also have a quick and dirty write-up for Magit window commands available here. Be warned that it’s far less comprehensive than this list, but you may find it helpful for basic navigation inside the window.

Last Updated: January 2022

Using Magit

Isaac Bell — Sat, 30 Oct 2021 20:22:26 GMT

This article is intended as a quick and dirty reference for the “Magit” Git interface in Doom Emacs.

For those unfamiliar with Magit itself, here is an expository blog post and an intro guide.

Several of the commands in this article are only available (by default) in Doom Emacs. To see a general reference guide for Magit, see the link below:

General Reference Sheet:

[Magit for Emacs
Edit descriptionkapeli.com](https://kapeli.com/cheat_sheets/Magit_for_Emacs.docset/Contents/Resources/Documents/index "https://kapeli.com/cheat_sheets/Magit_for_Emacs.docset/Contents/Resources/Documents/index")

What follows below is a basic set of commands to remember while navigating Magit in Doom Emacs.

Commands At a Glance:

Initialize Git Repository:

M-x magit-init

Navigate to the Magit Window:

SPC-g-g or C-x g

In the Magit Window

Select files or folders to stage, and stage them with the command S.

Unstage files with command u.

Once you’ve staged all files and folders you want, use command Cto create a commit.

Press TABto expand file diffs.

Enter a commit message, then close the commit buffer and save the commit using the Vim command :wqor :x.

To open a file, select the file and press ENTER.

Merging and Squashing

In the Magit Window, use the command M.

The following commands become available:

Close the merge actions buffer using C-g

Remotes

To bring up the Remotes window, press M in the Magit Window.

The following commands become available:

Pushing/Pulling

Push with the command p.

Pull with the command F.

Fetch with the command f.

Rails: CORS Preflight Options Requests

Isaac Bell — Fri, 30 Aug 2019 04:00:44 GMT

CORS header ‘Access-Control-Allow-Origin’ missing

You’re probably here because you’ve run into an error similar to the following response from your API call:

The request was redirected to 'https://example.com/foo', which is disallowed for cross-origin requests that require preflight

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://your-domain.com/endpoint (Reason: CORS header ‘Access-Control-Allow-Origin’ missing)

You will want to read the MDN docs on Preflight Requests to get acquainted with the reasons behind this issue.

You can resolve this issue using something like the following code in ApplicationController:

before_action :cors_set_access_control_headers

def cors_preflight_check
if request.method == 'OPTIONS'
cors_set_access_control_headers
render text: '', content_type: 'text/plain'
end
end

protected

def cors_set_access_control_headers
response.headers['Access-Control-Allow-Origin'] = '*'
response.headers['Access-Control-Allow-Methods'] = 'POST, GET, PUT, PATCH, DELETE, OPTIONS'
response.headers['Access-Control-Allow-Headers'] = 'Origin, Content-Type, Accept, Authorization, Token, Auth-Token, Email, X-User-Token, X-User-Email'
response.headers['Access-Control-Max-Age'] = '1728000'
end

Then, add a catch-all to your routes.rb file which matches all OPTIONS requests.

match '*all', controller: 'application', action: 'cors_preflight_check', via: [:options]

Sources:

https://gist.github.com/jpbalarini/54a1aa22ebb261af9d8bfd9a24e811f0

MDN CORS Primer: Preflight Requests

The Ruby Command Pattern — Pt. 2

Isaac Bell — Sat, 13 Jul 2019 18:16:09 GMT

Welcome back.

Let’s look at another example use of the Command Pattern in Ruby. This time, to give you a better idea of how it looks in practice (as opposed to the more conceptual explanation from the first article), we’ll make an extremely small-scale example.

We’ll use the Imperator gem to define a simple Command. Given a User model that handles an email, password, && an ID, we will set up a Command to create a new user. This command could be could be called from our model, controller, using background tasks, within other commands, etc.

Here is Imperator’s description of Commands:

Commands give you the opportunity to encapsulate all of the logic required for an interaction in one spot. Sometimes that interaction is as simple as a method call — more often there are several method calls involved, not all of which deal with domain logic (and thus, are inappropriate for inclusion on the models). Commands give you a place to bring all of these together in one spot without increasing coupling in your controllers or models.

For our sample code, let’s assume the command code is placed in an app/commands directory. Let's also assume we have a separate Notifierclass lying around which sends error logs.

**class** **Command::CreateUser** < Imperator::Command  
  **include** ActiveModel::Validations

 integer :id  
  string  :email  
  string  :password

 validates_presence_of :id  
  validates_presence_of :email  
  validates_presence_of :password

 **def** **action**  
    **if** !user.valid?  
      notify(  
        "Invalid attributes passed to Command::CreateUser",  
        user  
      )  
    **else**  
      user.save  
    **end**  
  **end**  

  **def** **notify**(error_message, user)  
    # This is just an example  
    Notifier.notify(error_message, user)  
  **end**  

  **def** **user**  
    @user ||= User.new.tap **do** |u|  
      u.id       = id  
      u.username = username  
      u.password = password  
    **end**  
  **end**  
**end**

Here’s how this code breaks down:

action is the main method which Imperator commands start from. We check if the attributes the command we received are valid, based on the validations set directly above.
If the attributes are not valid, we send out an error notification using our imaginary Notifier class.
If they are valid, we save the user. If you are doing something like synchronizing your data through an API, you might want to write a separate SaveUser command or similar to encapsulate that logic separately.
Depending on how complex your logic is going to be, you might take this and add a separate command for the Notifier or for error handling in general.

All of this is done without having to rely on a model or create a service with lots of moving parts. This is obviously extremely useful once you are working on a larger code-base with lots of moving parts. The Command structure can help you de-couple your logic from specific models, controllers, services, etc., by setting up designated tasks which handle specific and specialized responsibilities.

The Ruby Command Pattern — Pt. 1

Isaac Bell — Sat, 13 Jul 2019 18:13:54 GMT

An Asynchronous Back-End Processing Methodology

Using Rails, what if we need to encapsulate some side/background processing which runs completely independently of our web server MVC functions? We can use tools like Sidekiq, Resque, and Cron, yes. But from an architecture standpoint, how do we design our back-end processing workflow?

For a real-world example, let’s say we have a large-scale shopping application. While our application is busy handling customer transactions, account changes, pre-orders, and the like, we may also need to do extensive background processing so that we can store accurate records of all transactions to a private data store via, say, AWS. Plus, while this happens we may need to run regularly scheduled updates (let’s say one every three hours) to ensure that our data stays in sync between both sources.

The Gang of Four (four co-authors, that is) Design Patterns book illustrates lots of great examples of how to deal with various Object-Oriented pitfalls. Recommended reading for anybody interested.

The Command Pattern

One lesser-known design pattern, the Command Pattern, serves as a useful bridge. The Command Pattern is pretty simple, in reality it’s a very similar idea to closures in functional programming. It serves a somewhat similar use, as well. With Commands in Ruby we can encapsulate a self-contained request, worker, data transfer or any other sort of process as an object. This allows us to offload these operations into corresponding classes which can run on their own, concurrently or asynchronously. This Command can be designed to be equivalent to a pure function, or at least be written to produce minimal (if any) side effects outside of its own scope. It will run its own validations on the input it is given, and determine interactions without placing added responsibility on your data models or controllers.

Ultimately, not that different from closures.

In Practice — Background Processing Commands

Getting confused? Let’s make an example. Instead of starting of with a very basic example, I wanted to start with a real-world usage you might implement on a large-scale Rails application. If you want to start with the bare basics, go ahead and skip to part two.

Let’s say we have a store where users order resume-writing services. We have a class that we use to make API calls which retrieve records. We’ll use the Imperator gem to set up our Command structure.

Add this to your Gemfile: gem 'imperator'. This will also include the Virtusgem as a dependency, for declarative attribute handling.

# my_store_api.rb  
**require** 'rest-client'  
**require** 'restclient'

**class** **MyStoreAPI**  

    cattr_accessor :whatever_auth

 **class** << self  
        **def** **authorize**  
            # do some stuff  
            whatever_token = the_result  
        **end**

 # GET "https://fake-store.com/customers  
        **def** **get_customers**(options = {})  
            white_listed = %i(since name email city state)  
            get_with_parameters('/customers', whitelisted, options)  
        **end**

 # GET "https://fake-store.com/orders  
        **def** **get_orders**(options = {})  
            white_listed = %i(since resume_id author_id customer_id)  
            get_with_parameters('/orders', whitelisted, options)  
        **end**

 private

 **def** **get_with_parameters**(endpoint, whitelisted = [], options = {})  
          # Call our API endpoint  
        **end**  
    **end**  
**end**

Now, let’s say that every few hours, we need to do some processing and syncing on our back-end. We need to check which orders have successfully shipped, which are still pending approval, and which have been shipped but not received.

We can extract this into a Command which we will fire off in asynchronous queues. We can use any queue system such as Delayed Job, Resque, Sidekiq, etc. Here’s an example of what our Command can look like.

**class** **SyncOrdersFromAPI** < Imperator::Command  
  datetime since, default: Customer.not_synced.first.created_at  
  attribute should_log_orders, Virtus::Attribute::Boolean

 # This method required, we run our main function here  
  **def** **action**  
    orders.in_groups_of(1000).**do** |batch|  
      process_orders(batch)  
    **end**  
  **end**

 private

 **def** **orders**  
    @orders ||= MyStoreAPI.get_orders(since: since)  
  **end**

 **def** **sync_to_datastore**  
    process_order(order)  
    do_some_logging_on_order(order) **if** should_log_orders  
  **end**

 **def** **do_some_logging_on_order**(order)  
    # Logger.info(order)  
    # EmailNotifier.notify_order_synced(order)  
    puts "synced order #{order.inspect}"  
  **end**

 **def** **process_orders**(orders)  
    # do some processing and save to cold storage  
    orders.find_each { |order| sync_to_datastore(order) }  
  **end**  
**end**

Pretty straightforward, but very useful if you need to do a lot of validation and complicated logic such as batch processing, logging, syncing, or large background queries. We could add validations and dynamically suspend operations depending on what is going on in our core domain at runtime, by passing in relevant flags or runtime state details as attributes. In this way, we are able to keep our core application logic and our background processor logic isolated from one another. This is pretty straightforward and maintainable.

In our Sidekiq initializer we can call our Command like so:

Sidekiq.configure_server **do** |config|  
  config.periodic **do** |mgr|  
    # see **any** crontab **reference** **for** the **first** argument  
    # e.g. **http**://www.adminschoice.com/crontab-**quick**-**reference**  
    # **or**   https://crontab.guru/

 mgr.register(  
      '0 */3 * * *', # Runs every 3 hours  
      SyncOrdersFromAPI,   
      {   
        retry: 2,  
        queue: 'foo'  
      }  
    )  
  **end**  
**end**

Hopefully this gives you an idea of how to run Commands in Ruby/Rails, and how we can use this to set up isolated/encapsulated logic modules. This functionality can be expanded on to a very deep degree; let me know if you end up trying this out on your background processing system.

See you next time.

Removing QUnit from Your Ember Build

Isaac Bell — Sat, 01 Jun 2019 23:44:45 GMT

Tired of seeing Error: No tests were run in the console of your Ember application's dev environment?

Want to throw out QUnit from your Ember app? It’s not always so simple as deleting your /test folder, and it can be a bit of a pain in the side.

Well if you don’t, I don’t know why you’re here, but read on, I guess. We can do this in a simple series of steps. It’s pretty intuitive to remove QUnit using Grunt, so far as I can tell, and so for this tutorial we’ll assume you are struggling with this task using Bower and/or NPM includes.

Set “Test” Flag to False in Ember CLI Build

Modify your ember-cli-build.js like so:

module.exports = **function**(defaults) {  
   **var** app = **new** EmberApp(defaults, {  
+    tests: false,  
     'ember-cli-babel': {  
       presets: ["env"],  
       plugins: ["transform-es2015-arrow-functions"],

Note line 3.

Remove Imports from the Ember CLI Build

Remove any QUnit and Ember-QUnit imports from your ember-cli-build.jsi.e.:

- app.**import**('bower_components/qunit/qunit/qunit.js');  
- app.**import**('bower_components/ember-qunit/ember-qunit.amd.js');

Remove any of the following packages from your package.json and/or bower.json.

“qunit”
“ember-qunit”
“ember-cli-test-loader” [Optional]
“phantomjs” [Optional]
“ember-qunit-notifications”
“ember-cli-qunit”

Remove Any Test Scripts from your CI

Although this seems a bit counter-intuitive if you have one. (Are you sure you want to remove your test suite from your CI?)

script:  
   - npm run lint:hbs  
   - npm run lint:js  
   - npm test <-- **delete**

CLI Cleanup Command

Let’s clean our caches and get rid of un-used dependency folders. In other words, let’s refresh our modules folders.

$ bower **cache** clean && bower **install** --force && bower prune && rm -rf node_modules && npm cache clean --force && npm i

Let’s Debug: MetaSploit -“socket: Operation not permitted”

Isaac Bell — Sat, 01 Jun 2019 23:41:28 GMT

This is a breakdown of an error you might encounter with Metasploit depending on your Linux setup and user permissions.

Our Error

msf5 auxiliary(scanner/ip/ipidseq) > show options

Module options (auxiliary/scanner/ip/ipidseq):

 Name       Current Setting  Required  Description  
   ----       ---------------  --------  -----------  
   INTERFACE                   no        The name **of** the interface  
   RHOSTS     192.168.1.0/24   yes       The target address range **or** CIDR identifier  
   RPORT      80               yes       The target port  
   SNAPLEN    65535            yes       The number **of** bytes to capture  
   THREADS    50               yes       The number **of** concurrent threads  
   TIMEOUT    500              yes       The reply read timeout **in** milliseconds

msf5 auxiliary(scanner/ip/ipidseq) > run  
SIOCSIFFLAGS: Operation **not** permitted  
SIOCSIFFLAGS: Operation **not** permitted  
SIOCSIFFLAGS: Operation **not** permitted  
...

[-] Auxiliary failed: RuntimeError wlp3s0: You don't have permission to capture on that device (socket: Operation not permitted)  
[-] Call stack:  
[-]   /opt/metasploit-framework/lib/msf/core/exploit/capture.rb:124:in `open_live'  
[-]   /opt/metasploit-framework/lib/msf/core/exploit/capture.rb:124:**in** `open_pcap'  
[-]   /opt/metasploit-framework/modules/auxiliary/scanner/ip/ipidseq.rb:51:in `run_host'  
[-]   /opt/metasploit-framework/lib/msf/core/auxiliary/scanner.rb:111:in `block (2 levels) in run'  
[-]   /opt/metasploit-framework/lib/msf/core/thread_manager.rb:106:**in** `block in spawn'  
[*] Auxiliary module execution completed

Our Investigation

If you aren’t already doing so, you may want to test on your local IP 127.0.1.0 — you should still see the same error if all is as expected.

Relevant Github Issue: https://github.com/rapid7/metasploit-framework/issues/10721

Relevant AskUbuntu Issue: https://askubuntu.com/questions/530920/tcpdump-permissions-problem

This issue stems from either lack of permissions granted to the tcpdumptool, or issues with pcap installation in possible combination with missing setcap permissions.

Questions to Consider

Do you have tcpdump installed? In Linux try:

**ls** -la /usr/sbin | grep tcpdump

and see if you get some output similar to this.

-rwxr-x--- 1 root pcap 1130096 Mar 31  2018 /usr/sbin/tcpdump

Make sure you’ve given tcpdump the proper permissions it needs.

Also, you probably need to start running MSF as a root user if you aren’t already.

Resolution

Try these commands on Linux:

sudo setcap cap_net_raw,cap_net_bind_service=+eip $(which ruby)  
sudo setcap cap_net_raw,cap_net_bind_service=+eip $(which nmap)  
sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

and run MSF from scratch.

Again, are you a root user? If not, you will either need to run MSF as a root user (probably changing much of your configuration in the process) or you will need to change your user ID and group ID to 0 in your etcpassword file.

Keep in mind, it is extremely easy to lock yourself out of your computer messing around with this in particular, so make sure you really know what you’re doing and don’t edit this file while you’re logged in as the same user you’re editing. You could lose your sudo privileges at a very inopportune time, and find yourself stuck.

Further Information:

https://www.cyberciti.biz/faq/understanding-etcpasswd-file-format/

https://www.poftut.com/change-user-password-passwd-linux-etc-passwd-file/

Let’s Debug: Curb Gem Installation Failed

Isaac Bell — Sat, 01 Jun 2019 23:37:18 GMT

The Problem

You see this when you try to bundle your gems in a new Rails app.

**ERROR**:  Error installing curb:  
**ERROR**: Failed to build gem native extension.

/usr/local/rvm/rubies/ruby-2.1.2/bin/ruby extconf.rb  
checking **for** curl-config... no  
checking **for** main() **in** -lcurl... no

*** extconf.rb failed ***

Could **not** create Makefile due to some reason, probably lack **of** necessary  
libraries **and**/**or** headers.  Check the mkmf.log file **for** more details.  You may  
need configuration options.

The Cause

Windows users, I don’t have a solution for you. My apologies.

Curb depends on some native machine bindings which you’ll need to download as libraries. Specifically, it needs the libcurl3 and/or libcurl4libraries installed on your machine if you are on a Linux machine.

On OSX, the issue is a bit more complicated. See below.

The Solution

Mac (Updated as of 10/13/19)

Curb can no longer be installed via Homebrew, because it is no longer maintained by its author. This was previously the best way to install the gem on OS X.

[Delete curb.rb by core-code · Pull Request #54794 · Homebrew/homebrew-cask
You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or…github.com](https://github.com/Homebrew/homebrew-cask/pull/54794 "https://github.com/Homebrew/homebrew-cask/pull/54794")

The Github repository still exists, and it is possible to force wrestle it into your version of OS X, but understand that there is no official support from any vendors. Most likely, you will have to search through Github Issues for the gem to find a work-around for your Mac environment and OS Version.

Here are some leads to get you started:

Linux

First, run sudo apt-get update.

On Ubuntu:

sudo apt-**get** install libcurl4-gnutls-dev libcurl3 libcurl4-openssl-dev

On RedHat:

$ sudo yum install ruby-devel libcurl-devel openssl-devel

You may need to install the curl-devel package as well depending on your platform.

yum **install** curl-devel

Then try installation again.

If this helps you, or if you’d like to send over any corrections, feel free to reach out. Good luck from here on!

Troubleshooting Elastic Beanstalk Memory Issues

Isaac Bell — Mon, 25 Feb 2019 19:35:24 GMT

First question: Is it your logs?

Check your events, you may see something like this here

AWS EBS/EC2: No Disk Space Left on Device

If you came to this article with an emergency situation on your application here’s the quick and dirty: There are different reasons why your EC2 instance might run out of memory, but it is very common to run into this error because AWS logs are clogging up your instance.

You’re probably here because your Elastic Beanstalk environment’s Health Level went from OK to Severe. Frantically, you tried to download Full Logs to see where things went wrong, but the EBS console gave you the following error: Error: Rate Exceeded .

You only panicked for a few moments before you realized you could run eb logs from your terminal and poke around there. Or maybe you used eb ssh to log into your EC2 instance, and took a look at your logs only to realize that the logs themselves are the issue.

Here’s some quick steps towards a resolution.

Check Your Memory Usage

SSH into the EC2 instance and print out a breakdown of the memory usage.

Disk Space Check:

df [-h | -a]

Memory Usage Check:

du -ahx / | sort -rh | head -20

Check file sizes in a directory:

du -sh /var/app/current/log/*

If your results look something like this:

$ df -h

Filesystem Size Used Avail Use% Mounted on

devtmpfs 3.9G 60K 3.9G 1% /dev

tmpfs 3.9G 0 3.9G 0% /dev/shm

/dev/xvda1 7.8G 5.4G 2.4G 100% /

$ du -ahx / | sort -rh | head -20

7.6G /

2.3G /var

2.2G /opt

2.0G /var/app/current/log

2.0G /var/app/current

2.0G /var/app

2.0G /opt/rubies

1.5G /var/app/current/sidekiq.log

1.4G /var/app/current/log/production.log

1.2G /var/app/containerfiles/logs/rotated

(everything below under 1G)

$ du -sh /var/app/containerfiles/logs/rotated/*

7.0M production.log1551081661.gz

6.0M production.log1551085261.gz

39M production.log1551088861.gz

66M production.log1551092461.gz

66M production.log1551096061.gz

1.2G production.log1551099661

0 production.log1551103262

0 production.log1551106862

0 production.log1551110461

notice that the log files are way to big (in this case, the larger log files would be millions of lines long).

You’ll want to check the files to make sure that no critical info is in them. If it is, consider making a copy of the file to your local machine or copying relevant text from the file.

When you’re ready, remove all text from the files which are too large (DON’T delete them or you may encounter other issues).

sudo truncate -s 0 /var/app/current/log/production.log

sudo truncate -s 0 /var/app/current/log/sidekiq.log

sudo truncate -s 0 /var/app/containerfiles/logs/rotated/production.log1551099661

If space is completely full, you can’t download Full Logs from EBS, but you can remove one or two less-useful log files and then download Full Logs from the EBS Console before you remove the rest.

Once you’ve removed the files which need blasting, your EBS environment will likely begin working as it should. But, the Health will probably still show Severe. Redeploy your environment and you should be good to go.

Consider looking into Log Rotation Options for your environment, possibly by uploading to S3 using a service such as CloudWatch.

HTML Canvas Tutorial — Create and Manipulate 2D Particles

Isaac Bell — Fri, 11 Jan 2019 08:19:55 GMT

We’ve come a long way since the basics of the browser. Tools such as Three.js and WebGL Studio, animation libraries such as Raphael.js or GSAP, the growing popularity of SVGs, and the past decade’s increasingly conducive changes to Javascript have transformed animations in the browser from an occasional novelty to something real and fun to play with. The HTML canvas is a great way to get your feet wet creating directly interactive visuals in the browser. But there’s a lot to animation, and it can be best to start small. In this tutorial, we will learn how to make a simple particle cloud using a full-screen canvas, animate it, and make it respond to user input.

By the end of this tutorial, you will have created an element consisting of
small color particles which rotate around the mouse position; when the left mouse button is held down, the particle effect will scale out to circle the entire screen. Hopefully this article will give any Canvas newcomer some ideas to play around with.

Try the demo here: http://particle-vortex.herokuapp.com/

Read the full source code here: https://github.com/IsaacBell/Canvas-Particle-Vortex

Initial Variables

Let’s set up the initial conditions and variables for the effect first. This way, we can make global changes in one place. This is where we’ll set our necessary variables for the canvas element.

# Initial Setup

fps = 30   
width = window.innerWidth  
height = window.innerHeight  
r = 70  
scale = 1  
scaleMin = 12.5  
scaleMax = 100   
particleCount = 250  
canvas = undefined    # You can of course omit this line  
context = undefined   # Ditto  
particles = undefined # Ditto  
mouseX = width * 0.5  
mouseY = height * 0.5  
isMouseDown = false

We set a variable to store the size of the display window, we will pass this information to the canvas element in order to define its bounds. Variable r will determine the radius of the particle cloud. scale changes the size of the cloud, scaleMin allows us to set the absolute minimum scale of the cloud, and scaleMax allows you to set the maximum scale the cloud will reach when the user is holding the mouse down. particleCount allows you to increase or decrease the number of particles which appear the element; too many and we will run into performance issues. For the purpose of this demo, 30 FPS will work fine.

Our Init Function

**init** = ->  
  # Create canvas element  
  canvas = document.createElement('canvas')  
  canvas.id  = 'myCanvas'  
  document.body.appendChild(canvas)  
  **if** canvas **and** canvas.getContext  
    context = canvas.getContext('2d')  
    # Event handlers  
    window.addEventListener 'mousemove', onMouseMove, false  
    window.addEventListener 'mousedown', onMouseDown, false  
    window.addEventListener 'mouseup', onMouseUp, false  
    window.addEventListener 'onResize', onResize, false  
    createParticles()  
    onResize()  
    setInterval animLoop, 1000 / fps  
  **return**

Here’s what happening in the code here:

Step 1

We create a canvas element, and set it’s corresponding HTML element id to “myCanvas”. You will need an element with this ID in your HTML to match.

Step 2

We append (add) the canvas element it to the body of our HTML. If we miss this step, the canvas will never appear since it isn’t being drawn/rendered in the HTML document anywhere

Step 3

If the canvas has been properly set and we’re able to do setup using getContext()

-> We set our drawing context to ‘2D’

-> We add event listeners for mouse events and browser resizing

-> We call createParticles(), which we will use to generate our particle cloud

-> We call onResize(), which will fit our canvas element to its needed proportions

-> We set animLoop() to repeat at the rate of our fps variable

Color

The first function we’ll define is colorLuminance(). We need this for when we generate our particles; we’ll use this when we brighten or darken the hex colors we’ll generate as we create our particle cloud. colorLuminance() takes two arguments, the first being the hex string representing a color and second being a decimal between 1 and -1, indicating how much to brighten or darken the hex color.

For more details on what’s going on here, check out this article by Craig Buckler.

colorLuminance = (**hex**, lum) ->  
  # validate hex string  
  **hex** = String(**hex**).replace(/[^0-9a-f]/gi, '')   
  **if** hex.length < 6  
    **hex** = **hex**[0] + **hex**[0] + **hex**[1] + **hex**[1] + **hex**[2] + **hex**[2]  
  lum = lum **or** 0

 # convert to decimal and change luminosity  
  rgb = '#'  
  c = undefined  
  i = undefined  
  i = 0  
  **while** i < 3  
    c = parseInt(hex.substr(i * 2, 2), 16)  
    c = Math.round(Math.min(Math.max(0, c + c * lum), 255)).toString(16)  
    rgb += ('00' + c).substr(c.length)  
    i++  
  **return** rgb

With that out of the way, let’s make the function that will serve as much of the meat of our project.

Creating Our Particles

We’ll use a while loop to create an array of particles with randomized movement speeds, fill color, and orbit distance from the center of the cloud. The fillColor value makes use of the colorLuminance() function we just added previously; play around with this line to find a color range you like. The speed and orbit distance of each particle is randomized to an extent, by multiplying values times the value returned by Math.random(). This is a very common pattern in graphics/color processing.

createParticles = ->  
  particles = []  
  i = 0  
  **while** i < particleCount  
    particle =   
      size: 5  
      position:  
        x: mouseX  
        y: mouseY  
      offset:  
        x: 0  
        y: 0  
      shift:  
        x: mouseX  
        y: mouseY  
      speed: 0.02 + Math.random() * 0.02  
      targetSize: 1  
      fillColor: colorLuminance('#' + Math.random().toString(16), -0.23)  
      orbit: r / 3 * Math.random()  
    particles.push particle  
    i++  
  **return**

Responding to Event Listeners

Next we’ll write the functions which will be triggered by the event listeners we set up earlier. Nothing too complicated, we simply store the mouse’s dynamic position and mouseDown status in our top-level variables. Lastly, when the browser window is resized, we will change the dimensions of the canvas element.

# Simple event Listener functions

**onMouseMove** = (e) ->  
  mouseX = e.clientX - ((window.innerWidth - width) * .5)  
  mouseY = e.clientY - ((window.innerHeight - height) * .5)  
  **return**

**onMouseDown** = ->  
  isMouseDown = true

**onMouseUp** = ->  
  isMouseDown = false

**onResize** = ->  
  width = window.innerWidth  
  height = window.innerHeight  
  canvas.width = width  
  canvas.height = height  
  **return**

The Animation Loop

Now for the render loop itself. This is where the real action is; don’t get intimidated looking at it.

Here’s a breakdown of what we’re doing:

We scale the particle cloud to its maximum or minimum size depending on whether the mouse is down
With context.fillStyle() we set a color to draw shapes in. We choose black, with 55% opacity
With context.fillRect() we draw up the bounds for the canvas rectangle we’ll be filling in
Using a simple while loop, we iterate through the particles and rotate, shift, and gently enlarge each as needed. The faster the fps variable is, the faster these shifts and rotations will appear to the eye
Once we’ve calculated the new size and position of our particle in each iteration, we use the canvas api to draw our circle

**animLoop** = ->  
  **if** isMouseDown  
    # Expand the cloud when mouse is clicked down  
    scale += (scaleMax - scale) * 0.2  
  **else**  
    # Or else shrink the cloud down  
    scale -= (scale - scaleMin) * 0.2  
  # Apply whichever change we set above  
  scale = Math.min(scale, scaleMax)

 # Set our line opacity and limit our drawing board   
  # size to the size of the screen  
  context.fillStyle = 'rgba(0,0,0,0.55)'  
  context.fillRect 0, 0, context.canvas.width, context.canvas.height  

  i = 0  
  len = particles.length  

  **while** i < len  
    particle = particles[i]  

    # Rotation  
    particle.offset.x += particle.speed * 0.7  
    particle.offset.y += particle.speed * 0.7  

    # Follow the mouse, with a bit of blur/lag effect  
    particle.shift.x += (mouseX - (particle.shift.x)) * particle.speed * 0.6  
    particle.shift.y += (mouseY - (particle.shift.y)) * particle.speed * 0.6  

    # Shift the particles accordingly, using a cosine function  
    particle.position.x = particle.shift.x + Math.cos(i + particle.offset.x) * particle.orbit * scale  
    particle.position.y = particle.shift.y + Math.sin(i + particle.offset.y) * particle.orbit * scale  

    # Limit our animation to the screen bounds  
    particle.position.x = Math.max(Math.min(particle.position.x, width), 0)  
    particle.position.y = Math.max(Math.min(particle.position.y, height), 0)  
    particle.size += (particle.targetSize - (particle.size)) * 0.05  

    **if** Math.round(particle.size) == Math.round(particle.targetSize)  
      particle.targetSize = 1 + Math.random() * 10  

    # Finally, let's do some drawing!  

    context.beginPath()  

    # Select line color at random  
    context.fillStyle = particle.fillColor  
    context.strokeStyle = particle.fillColor  
    context.lineWidth = particle.size  
    context.moveTo particle.position.x, particle.position.y  
    context.lineTo particle.position.x, particle.position.y  
    context.stroke()  
    context.arc particle.position.x, particle.position.y, particle.size / 2, 0, Math.PI * 2, true  
    context.fill()  
    i++  

  **return**

Lastly, we need to call our initializer function to jump things off.

window.onload = init

Conclusion

Hopefully you can get an idea of how to bring fully interactive visuals to a web browser using the canvas. This demo could easily be expanded, optimized, or modified. There’s lots of potential. Play around with the variables, download the source on Github, experiment in your browser.

Github Repo: https://github.com/IsaacBell/Canvas-Particle-Vortex

Introducing Twilio

Isaac Bell — Fri, 11 Jan 2019 07:46:50 GMT

Introducing Twilio

Twilio is a powerful cloud communications platform which provides an interface for SMS, voice call utilities, answering machines, virtual phone lines, and more. With this platform we can build a virtual communications system which we can then count on to remain reliable in our applications as we deploy them to over 150 countries worldwide.

We’ll build a simple and easily extensible Ruby integration for the Twilio core platform, then use that to perform a few tasks:

Sending SMS with Ruby
Verifying phone numbers via the Lookup API.
Then we’ll look at how to set up two-factor authentication using this service.

We will not cover making calls with the service today, but that may be something I cover in the future if requested.

For the purpose of this tutorial we will be using Ruby on Rails.

You Will Need:

A Twilio account
Twilio test credentials including: Your Twilio Account SID Your Authentication Token
At least one active Twilio phone number with Messaging enabled
[For Step 2] The Twilio Lookup API enabled

General Tips

Twilio has some hard rules as to their phone system. All numbers follow the E.164 international format code. To ensure our phone numbers follow this standard, we will use the Phony gem.
For international messaging, restrictions apply for some countries. A phone line from a particular country may be limited to sending only in that country or select others. To deal with this, I recommend buying multiple phone lines in order to cover all of the countries you need to. Twilio phone numbers (as I write this article in Q3 2018) are only $1, so it’s simple and cost-effective to get two or three to split sending between.

If you are new to the platform, you should check whether phone lines are available in the area you want with the capabilities you need. You should also read up on Twilio’s International Messaging support page to see if there are any gotchas for your country.

Getting Started

First, let’s add the Twilio Ruby gem to our Ruby app. From your terminal, run:

$ bundler add twilio-ruby

Or append gem 'twilio-ruby' to your Gemfile.

Building Our Twilio Integration

Base Twilio Service

Now, let’s start on our Twilio interface. First we will set up our base class which connects to the Twilio API.

Here we will implement the common Service pattern to abstract the Twilio API connection. For this tutorial, we will create two sub-classes which pull from this base. We can then easily re-use this code from within any model or controller class.

**require** 'twilio-ruby'

**class** **Services::Twilio**  
  cattr_accessor :client_adapter  
  **self**.client_adapter = Twilio::REST::Client

 **attr_accessor** :client, :t_phone  

  **attr_reader** :user # Optional

 **def** **initialize**(opts = {})  
    # Here you might stop initialization or raise an error if   
    # certain options are missing from the argument hash  
    # i.e. `raise SomeError unless opts[:to].present?`

 # Optional  
    @user = opts[:user]  

    # Twilio Credentials  
    @account_sid = 'ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'  
    @auth_token  = 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'  
    @client  = **self**.**class**.client_adapter.new(@account_sid, @auth_token)  

    # Our message/payload and its recipient  
    @body = opts[:body]  
    @t_phone = opts[:to]  
  **end**  

  protected

 # Base message sending method  
  **def** **send**(options = {})  
    # Either leave this section blank, or set up a default send action  
    # for your child service classes to inherit  

    # Here is an example of a default action -  
    # This base method will attempt to decide   
    # whether to make a call or text based on  
    # input args, then attempt to send the payload  
    options.reverse_merge!({  
      to: @t_phone,  
      body: @body,  
      from: @from  
    }) **unless** @t_phone.blank?  

    **if** options[:url].present? || options[:record]  
      # These options will only be present for a call  
      @client.api.account.calls.create(options)  
    **else**  
      @client.api.account.messages.create(options)  
    **end**  
  **end**  
**end**

First, we set up the client_adapter class attribute. This will be important because when we want to test our service, we can easily change this adapter to a mock version. Or, for your setup, you might like to call some other service if you are in a custom environment, say staging. More details on this later.

In the initialize() function we create a new instance of the Twilio client using our Twilio SID and authentication token.

The send() function here is just a parent method for child classes to pull from. Omit it at will without error. We are probably never going to call Services::Twilio.send() naked, but by adding this method to the parent class we can set up a default action for any of our child classes.

As it is written here, the parent send() method defaults to sending an SMS, and this will be the behavior of any Twilio service which inherits from this base class and doesn’t have its own send() method defined. You can tailor the default action to your needs, or omit it to get an error when send() is not defined in our child classes.

Twilio SMS Service

We will build a child class for each Twilio product we want to harness. First, let’s take a look at SMS delivery. For this, we will create a sub-class of our Twilio Base Service. This class is straightforward at its core; we will first override the send() method from our base class.

Below is a basic setup you can start from; using this setup, we take our options hash, fill in any blank parameters with whatever relevant data the service object was initialized with, and pass the parameters to the Twilio API.

**require** 'twilio-ruby'

**class** **Services::Twilio::SMS** < Services::Twilio  
  **def** **send**(options = {})  
    options.reverse_merge!({  
      :from => @from,  
      :to   => @to || @t_phone,  
      :body => @body  
    })  
    @client.api.account.messages.create(options)  
  **end**  
**end**

Extending Our SMS Service

Below is an example of some ways you can extend this SMS service to handle your application’s needs. You may need to adjust some of this code based a few factors.

First, you may need to write a method to properly check and format the phone number we’ll be sending to. Using our receiving_number method, we can pull our target phone number from any data source we need to. Here, for example purposes, I have added a check for a user model with a sms_number attribute set. Edit or omit this line as needed. Here we also use .phony_formatted() to ensure that we follow E.164 protocol mentioned earlier.

Second, if you are sending from multiple phone lines there are some considerations to take into account. If you bought multiple lines because you need to get around international sending restrictions (see General Tipsabove), you will add logic here to handle that. These laws change all the time, so make sure you check Twilio’s international messaging guidelinesto see what logic you need to add. Or, you may prefer to send your SMS through a phone line from the same country as the phone which will be receiving the text.

For example, we could send from a Malaysian phone line (Malaysian lines currently in beta) when sending to Malaysian numbers, but otherwise send from a default US number.

For some countries (France and India, others), you will probably want to add a check to determine whether you are sending transactional emails (alerts, two-factor auth codes, etc.) or marketing emails and adjust your code accordingly, but for simplicity we won’t cover that logic here.

**require** 'twilio-ruby'

**class** **Services::Twilio::SMS** < Services::Twilio  
  **def** **send**(options = {})  
    options.reverse_merge!({  
      :from => @from || sending_number,  
      :to   => receiving_number,  
      :body => @body  
    })  
    @client.api.account.messages.create(options)  
  **end**  

  **def** **receiving_number**  
    **if** @user.present? && !@user.sms_number.blank?  
      # Example of a phone number pulled from a user model  
      @user.sms_number.phony_formatted(format: :international)  
    **else**  
      @t_phone.phony_formatted(format: :international)  
    **end**  
  **end**  

  **def** **sending_number**  
    # Use one phone line for US and UK sending, and the other for int'l  
    # This method can be overriden in our user model when we call this service  
    sending_number = **if** receiving_number.**include**?('+1') || receiving_number.**include**?('+55')  
      ENV['TWILIO_US_NUM']  
    **else**  
      ENV['TWILIO_I8LN_NUM']  
    **end**  
  **end**  
**end**

Now our service is looking pretty beefy.

That does it on our SMS sending. Now let’s take a look at the Twilio Phone Lookup API. Here again we inherit from the base Twilio service.

Twilio Phone Lookup

We will borrow from our Twilio base class again, but for this part of the API we don’t need to fire a send() method. Instead, we will define a fetch()method which will make a retrieval call to the Twilio API. I’ve also make the send() method an alias of fetch() so no confusion occurs as you expand on this code; some of you will prefer to alias the method to search(input_number) instead.

Note the catch_twilio_404() method. This block is crucial because the Twilio API will return an error response (specifically a Twilio::Rest::RestError )if the phone lookup doesn’t find a match. Instead of returning a naked exception and possibly crashing our application with it, we catch any such error as soon as it occurs and return false instead. I recommend using a gem such as Exception Notification or Paper Trail to log these errors when they occur.

# example = Services::Twilio::PhoneLookup.new(user: current_user)  
# example.lookup  
# example.search('+11234567890')

**class** **Services::Twilio::PhoneLookup** < Services::Twilio  
  # Use this is you are pulling your phone number from a user model  
  **def** **fetch**  
    lookup_result = catch_twilio_404 **do**  
      address_book(@user.two_factor_sms).fetch  
    **end**  
    # This will either resolve to false or to a Twilio API Lookup object.  
    lookup_result  
  **end**

 # Use this to manually enter a number for lookup  
  **def** **search**(input_number)  
    lookup_result = catch_twilio_404 **do**  
      address_book(input_number).fetch  
    **end**  
    # This will either resolve to false or to a Twilio API Lookup object.  
    lookup_result  
  **end**

 # This is where we actually run the API call for the lookup  
  **def** **address_book**(number_to_search)  
    @client.lookups.v1.phone_numbers(number_to_search)  
  **end**

 # Wrap this block around our lookup in order to  
  # ensure we don't run any into errors in our application  
  **def** **catch_twilio_404**  
    **begin**  
      **yield**  
    **rescue** Twilio::REST::RestError => err  
      # Add error handling/notification/logging code here  
      # Then we return false instead of an exception  
      false  
    **end**  
  **end**

 **def** **lookup**  
    fetch  
  **end**

 **def** **send**  
    fetch  
  **end**  
**end**

Using the Phone Lookup In A User Model to Validate Numbers

Again, rename the two_factor_sms attribute to whatever name you are working with in your model. This implementation requires the phony_railsgem.

$ rails g migration AddTwoFactorToUsers two_factor_sms:**string** two_factor_enabled:**string**

# user.rb

# Callbacks  
validates_plausible_phone :two_factor_sms, allow_blank: true  
phony_normalize :two_factor_sms, default_country_code: 'AU'  
validate :valid_sms_number

# Custom callback - validates our phone number through Twilio  
**def** **valid_sms_number**  
  **if** !Rails.env.test? && **self**.two_factor_sms.present?      
    lookup = Services::Twilio::PhoneLookup.new(**self**)  
    **if** lookup.send == false  
      errors.add(:two_factor_sms, 'is not a valid phone number')  
    **end**  
  **end**  
**end**

Test Suite

Finally, let’s test our integration. To do this, we need some test doubles which will emulate the functionality of calling the Twilio API itself. And so, we are going to create a mock object for our SMS and stub the API calls in our Phone Lookup service.

Here we are going to build off of Thoughtbot’s test implementation of the Twilio service.

Stubbing Our Phone Lookup
By overriding most of the methods in the Twilio call chain to instead return self, we bypass the errors we would get if those methods actually ran in our test environment. This way we don’t have to make any changes to our application code to bypass that issue, like a begin/rescue to catch specific test mode errors or similar. We avoid a lot of messy code bloat this way.

**class** **FakePhoneLookup**  
  Carrier = Struct.new(:type, :name)

 SMS_LOOKUP_HASH = {  
    '+12815552134' => Carrier.new('mobile', 'T-Mobile USA, Inc.'),  
    '+18475552134' => Carrier.new('landline', 'AT&T')  
  }

 cattr_accessor :return_type, :lookups, :carrier, :numbers  
  **attr_accessor**  :fetch, :num

 **self**.lookups = []  
  **self**.numbers = []

 # def initialize(_account_sid, _auth_token)  
  #   self.carrier = nil  
  # end

 **def** **phone_numbers**  
    **self**  
  **end**

 **def** **v1**  
    **self**  
  **end**

 **def** **lookup**(num)  
    @fetch = true  
    true  
  **end**  
  **def** **lookup**(num, true_or_false = true)  
    true_or_false  
  **end**

 **def** **lookups**  
    **self**  
  **end**

 **def** **send**  
    true  
  **end**

 **def** **address_book**(sms)  
    **self**  
  **end**  
  **def** **phone_numbers**(sms)  
    **self**  
  **end**

 **def** **catch_twilio_404**  
    **yield**  
  **end**

 **def** **client**  
    **self**  
  **end**

 **def** **num**  
    **self**.numbers.last  
  **end**

 **def** **account**  
    **self**  
  **end**

 **def** **sms**  
    **self**  
  **end**

 **def** **api**  
    **self**  
  **end**

 **def** **messages**  
    **self**  
  **end**  
**end**

Mock SMS Messages

**class** **FakeSMS**  
  SmsMessage = Struct.new(:from, :to, :body)

 cattr_accessor :messages  
  **self**.messages = []

 **def** **initialize**(_account_sid, _auth_token)  
  **end**

 **def** **num**  
    messages[-1].to  
  **end**

 **def** **account**  
    **self**  
  **end**

 **def** **sms**  
    **self**  
  **end**

 **def** **lookups**  
    FakePhoneLookup.new  
  **end**

 **def** **api**  
    **self**  
  **end**

 **def** **messages**  
    **self**  
  **end**

 **def** **create**(from:, to:, body:)  
    **self**.**class**.messages << SmsMessage.new(from, to, body)  
  **end**  
**end**

Adding Our Test Doubles to The Test Suite

Add something like to following to your Rspec configuration.

# spec_helper.rb  
RSpec.configure **do** |config|  
  config.before(:each) **do** |example|  
    Services::Twilio.client_adapter = FakeSMS  
    Services::Twilio::SMS.client_adapter = FakeSMS

 Services::Twilio::PhoneLookup.stubs(:fetch).returns(true)  
    Services::Twilio::PhoneLookup.stubs(:lookup).returns(true)  
    Services::Twilio::PhoneLookup.stubs(:send).returns(true)

 **end**

 config.after(:each) **do**  
    # Reset our mock messages after test finishes  
    FakeSMS.messages = []  
  **end**  
**end**

Two-Factor Authentication Using One-Time Passwords

Now that we have our model layer fully set up, let’s use our new services to power a two-factor authentication system.

Adding One-Time Passwords and SMS methods to Our User Model
Let’s add a simple ROTP functionality to our user model. It’s not terribly hard to implement this ourselves, but let’s look at how we would do this using the active_model_otp gem.

To see another implementation of this gem, take a look at https://coderwall.com/p/qw7hwq/effortless-two-factor-authentication-in-rails.

Here’s an implementation that will work for our purposes. Tweak to your needs.

Our migration:

$ rails g migration AddOtpSecretKeyToUsers otp_secret_key:string

User Model Additions:

# user.rb  
has_one_time_password

**def** **send_sms_auth**  
  **if** **self**.two_factor? && **self**.two_factor_keys_present?  
    receiving_number = **self**.sms_number.phony_formatted(format: :international)

 # You can override the Twilio service here  
    sending_number = ENV['ALTERNATE_TWILIO_PHONE_NUMBER']

 **return** Services::Twilio::SMS.new(**self**, {  
      from: sending_number, # optional  
      to: receiving_number,  
      body:  "Your code is #{**self**.otp_code}"  
    }).send

 **else**  
    **return** false  
  **end**  
**end**

**def** **sms_number**  
  **self**.two_factor_sms # || self.company.phone || whatever else  
**end**

**def** **set_sms**(sms)  
  enable_two_factor  
  **self**.two_factor_sms = sms  
  **self**.save  
**end**

**def** **enable_two_factor**  
  **self**.two_factor_enabled = true  
**end**

**def** **disable_two_factor**  
  **self**.two_factor_enabled = false  
**end**

**def** **verify_sms_auth**(sms)  
  authenticate_otp_code(sms) || authenticate_otp(sms, drift: 600)  
**end**

**def** **reset_one_time_password**  
  **self**.otp_code = 6.times.map{rand(10)}.join # or whatever  
**end**

**def** **set_one_time_password_secret**  
  # self.otp_regenerate_secret  
  **self**.otp_secret_key = ROTP::Base32.random_base32  
**end**

**def** **ensure_has_one_time_password**  
  **self**.set_one_time_password_secret   **if** **self**.otp_secret_key.blank?  
  **self**.reset_one_time_password        **if** **self**.otp_code.blank?  
  **yield**  
**end**

**def** **valid_sms_number**  
  **if** !Rails.env.test? && **self**.two_factor? && two_factor_sms_changed?  
    lookup = Services::Twilio::PhoneLookup.new(**self**)  
    **if** lookup.send == false  
      errors.add(:two_factor_sms, 'is not a valid phone number')  
    **end**  
  **end**  
**end**

**def** **authenticate_otp_code**(otp)  
  otp == **self**.otp_code  
**end**

**def** **authenticate_otp_secret**(otp)  
  otp == **self**.otp_secret_key  
**end**

routes.rb
Let’s add routes for our simple two-factor auth.

match '/sms-auth' => 'twilio#send_sms_auth_code',  
      as: :send_sms_auth_code, via: [:post, :get]  
post '/sms-verify' => 'twilio#verify_sms_auth_code',  
      as: :verify_sms_auth_code

Twilio Controller Spec
Before we get to the controller addition, I wanted to show you how the spec would look in practice. Here it is, using our FakeSMS mock client.

require 'spec_helper'

**describe** TwilioController, **type**: :controller **do**  
  let(:sms_mock_client) { FakeSMS }  
  let(:**user**) { FactoryBot.create(:**user**, {  
    two_factor_sms: '+12345678900',  
    two_factor_enabled: true  
  })}  
  subject { **user** }

 **describe** "[GET] send_sms_auth_code" **do**  
    it "redirects to form to enter code" **do**  
      **get** :send_sms_auth_code, **id**: user.id  
      expect(response.code).to eq '200'  
    **end**

 it "doesn't accept user without two-factor enabled" **do**  
      user2 = FactoryBot.create(:**user**,  
        email: 'abc@xyz.com',  
        two_factor_enabled: false  
      )  
      **get** :send_sms_auth_code, **id**: user2.id  
      expect(response).to redirect_to new_user_session_path  
    **end**  
  **end**

 **describe** '[POST] verify_sms_auth_code' **do**  
    it 'signs in user with correct code' **do**  
      params = { **id**: user.id, sms: user.otp_code, secret: user.otp_secret_key }  
      post :verify_sms_auth_code, params  
      expect(response).to redirect_to messages_path  
    **end**  
    it 'fails with incorrect sms' **do**  
      params = { **id**: user.id, sms: '111111', secret: user.otp_secret_key }  
      post :verify_sms_auth_code, params  
      expect(response).not_to redirect_to messages_path  
    **end**  
    it 'fails with incorrect secret key' **do**  
      params = { **id**: user.id, sms: user.otp_code, secret: '1234321' }  
      post :verify_sms_auth_code, params  
      expect(response).not_to redirect_to messages_path  
    **end**  
  **end**  
**end**

Twilio Controller
Now, let’s add our code to get those tests passing. We add our GET route to send out the code (yes I made it a GET Request, spew your criticism in the comments) and our POST route to verify the code.

**class** **TwilioController** < ApplicationController  
  # GET "/send-sms-auth-code"  
  **def** **send_sms_auth_code**  
    @user = resource_type.find(params[:id])

 **if** @user.send_sms_auth  
      flash[:notice] = "SMS sent to #{@user.two_factor_sms}"  
      # render :two_factor_form  
    **else**  
      redirect_to new_user_session_path,  
        notice: "SMS not enabled for #{@user.email}"  
    **end**  
  **end**

 # POST "/sms-verify"  
  **def** **verify_sms_auth_code**  
    @user = resource_type.find(params[:id])  
    key_matches = params[:secret].present? && @user.otp_secret_key == params[:secret]

 **if** key_matches && params[:sms].present? && @user.verify_sms_auth(params[:sms])  
      sign_in(@user, bypass: true)  
      redirect_to after_sms_verification_path  
    **else**  
      redirect_to new_user_session_path,  
        notice: 'Failed authentication, please try again.'  
    **end**  
  **end**

 protected

 **def** **admin?**  
    params['admin'] == true || params['admin'] == 'true'  
  **end**

 **def** **after_sms_verification_path**  
    admin? ? auth_users_path : messages_path  
  **end**

 **def** **resource_type**  
    admin? ? AdminUser : User  
  **end**  
**end**

Making Our Tests End-to-End

Integration Spec
You’ll need to tweak this to match your domain and environment and such, but this is a great addition to have in your test suite. The template goes like this:

# spec/integration/sms_spec.rb

**require** 'spec_helper'

describe 'SMS', type: :request **do**  
  let(:sms_mock_client) { FakeSMS }  
  let(:user) { FactoryBot.create(:user, {  
    two_factor_sms: '+12345678900',  
    two_factor_enabled: true  
  })}  
  subject { sms_mock_client }

 describe '[POST] send_sms_auth_code' **do**

 it 'sends a text message via the Twilio API after a notication is created' **do**  
      headers = {  
        "ACCEPT" => "application/json",  
        'HTTP_REFERER' => 'http://example.com'  
      }

 get "/sms-2fa/",  
      {  
        id: user.id,  
        secret: user.otp_secret_key  
      }, headers

 expect(subject.messages.count).to eq 1  
      expect(subject.messages[0].from).to eq ENV['TWILIO_NUM']  
      expect(subject.messages[0].to).to eq '+12345678900'  
      expect(subject.messages[0].body).not_to be_nil  
    **end**  
  **end**  
**end**

Feature Spec
You’ll probably need to tweak this as well depending on how you set up your front end, but this test is extremely helpful to include as well.

require 'spec_helper'

feature 'Two-Factor Auth' **do**  
  let(:**user**) { FactoryBot.create(:**user**,  
    **password**: 123456,  
    two_factor_sms: '+14043548389',  
    two_factor_enabled: true  
  )}  
  let(:**admin**) { FactoryBot.create(:admin_user,  
    **password**: 'pass123456',  
    two_factor_sms: '+12344234495',  
    two_factor_enabled: true  
  )}

 scenario 'Send one-time password code on default login' **do**  
    previous_sms_count = FakeSMS.messages.count

 visit('/login')  
    fill_in 'user_email', **with**: user.email  
    fill_in 'user_password', **with**: '123456'  
    click_button 'Sign in'

 page.should have_content('SMS Authentication')

 expect(FakeSMS.messages.count).to eq previous_sms_count + 1

 fill_in 'sms', **with**: user.otp_code  
    click_button 'Verify SMS code'

 **within** '.site-navigation' **do**  
      page.should have_content('Manage')  
    **end**  
  **end**

 scenario 'Send one-time password code on admin login' **do**  
    previous_sms_count = FakeSMS.messages.count

 visit('/2fa/login')  
    fill_in 'admin_user_email',    **with**: admin.email  
    fill_in 'admin_user_password', **with**: 'pass123456'  
    click_button 'Login'

 expect(page.title).not_to **include** 'Login'  
    page.should have_content('SMS Authentication')  
    expect(FakeSMS.messages.count).to eq previous_sms_count + 1

 fill_in 'sms', **with**: admin.otp_code  
    click_button 'Verify SMS code'

 page.should have_content('My Dashboard')  
    page.should have_content(admin.email)  
  **end**  
**end**

Final Thoughts

I hope that this helped you understand at least the gist of setting up Two-Factor Authentication in Rails. If you are able to set up a working implementation based on this, I’d love to hear from you with your thoughts. This was my first technical tutorial to go this far in-depth, please let me know if you need clarifications or help repeating this processing on your own application. More to come.