text:
# Parsing and indexing PDF in Python

I have a Doxie Go scanner and I scan all the documents I receive in paper. That's nice, but it creates another problem. All the resulting PDF files have to be named, organized and stored... Doing that manually is boring and time consuming. Of course that's something I want to automate!

I even bought Hazel a while ago. It's a nice software that monitors files in a folder and performs specific instructions based on the rules you defined. It works well but I felt a bit limited and I thought I could probably write something more tailored to my use case. And that would be more fun :-)

## Parsing PDF in Python¶

A quick solution I found was to run pdftotext using subprocess. I looked at PDFMiner, a pure Python PDF parser but I found pdftotext output to be more accurate. On MacOS, you can install it using Homebrew:

$brew install Caskroom/cask/pdftotext Here is a simple Python function to do that: In [1]: import subprocess def parse_pdf(filename): try: content = subprocess.check_output(["pdftotext", '-enc', 'UTF-8', filename, "-"]) except subprocess.CalledProcessError as e: print('Skipping {} (pdftotext returned status {})'.format(filename, e.returncode)) return None return content.decode('utf-8')  Let's try to parse a pdf file. We'll use requests to download a sample file. In [2]: import requests url = 'http://www.cbu.edu.zm/downloads/pdf-sample.pdf' response = requests.get(url) with open('/tmp/pdf-sample.pdf', 'wb') as f: f.write(response.content)  Let's first look at the PDF: In [3]: from IPython.display import IFrame IFrame('http://www.cbu.edu.zm/downloads/pdf-sample.pdf', width=600, height=870)  Out[3]: Nothing complex. It should be easy to parse. In [4]: content = parse_pdf('/tmp/pdf-sample.pdf') content  Out[4]: "Adobe Acrobat PDF Files\nAdobe® Portable Document Format (PDF) is a universal file format that preserves all of the fonts, formatting, colours and graphics of any source document, regardless of the application and platform used to create it. Adobe PDF is an ideal format for electronic document distribution as it overcomes the problems commonly encountered with electronic file sharing. • Anyone, anywhere can open a PDF file. All you need is the free Adobe Acrobat Reader. Recipients of other file formats sometimes can't open files because they don't have the applications used to create the documents. PDF files always print correctly on any printing device. PDF files always display exactly as created, regardless of fonts, software, and operating systems. Fonts, and graphics are not lost due to platform, software, and version incompatibilities. The free Acrobat Reader is easy to download and can be freely distributed by anyone. Compact PDF files are smaller than their source files and download a page at a time for fast display on the Web.\n\n• •\n\n• •\n\n\x0c" This works quite well. The layout is not respected but it's the text that matters. It would be easy to define some regex to define rules based on the PDF content. This could be the first step in naming and organizing the scanned documents. But it would be nice to have an interface to easily search in all the files. I've already used MongoDB full text search in a webapp I wrote and it worked well for my use case. But I read about Elasticsearch and I always wanted to give it a try. ## Elasticsearch Ingest Attachment Processor Plugin¶ I could just index the result from pdftotext, but I know there is a plugin that can parse PDF files. The Mapper Attachments Type plugin is deprecated in 5.0.0. It has been replaced with the ingest-attachment plugin. So let's look at that. ### Running Elasticsearch¶ To run Elasticsearch, the easiest is to use Docker. As the official image from Docker Hub comes with no plugin, we'll create our own image. See Elasticsearch Plugin Management with Docker for more information. Here is our Dockerfile: FROM elasticsearch:5 RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-attachment Create the elasticsearch-ingest docker image: $ docker build -t elasticsearch-ingest .

We can now run elasticsearch with the ingest-attachment plugin:

$docker run -d -p 9200:9200 elasticsearch-ingest ### Python Elasticsearch Client¶ We'll use elasticsearch-py to interact with our Elasticsearch cluster. In [5]: from elasticsearch import Elasticsearch es = Elasticsearch()  Let's first check that our elasticsearch cluster is alive by asking about its health: In [6]: es.cat.health()  Out[6]: '1479333419 21:56:59 elasticsearch green 1 1 0 0 0 0 0 0 - 100.0%\n' Nice! We can start playing with our ES cluster. As described in the documentation, we first have to create a pipeline to use the Ingest Attachment Processor Plugin: PUT _ingest/pipeline/attachment { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "data" } } ] } OK, how do we do that using the Python client? In [7]: body = { "description" : "Extract attachment information", "processors" : [ { "attachment" : { "field" : "data" } } ] } es.index(index='_ingest', doc_type='pipeline', id='attachment', body=body)  Out[7]: {'acknowledged': True} Now, we can send a document to our pipeline. Let's start by using the same example as in the documentation: PUT my_index/my_type/my_id?pipeline=attachment { "data": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=" } Using Python client, this gives: In [8]: result1 = es.index(index='my_index', doc_type='my_type', pipeline='attachment', body={'data': "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0="}) result1  Out[8]: {'_id': 'AVhvJKzVIvjFWZACJU_t', '_index': 'my_index', '_shards': {'failed': 0, 'successful': 1, 'total': 2}, '_type': 'my_type', '_version': 1, 'created': True, 'result': 'created'} Let's try to get the created document based on its id: In [9]: es.get(index='my_index', doc_type='my_type', id=result1['_id'])  Out[9]: {'_id': 'AVhvJKzVIvjFWZACJU_t', '_index': 'my_index', '_source': {'attachment': {'content': 'Lorem ipsum dolor sit amet', 'content_length': 28, 'content_type': 'application/rtf', 'language': 'ro'}, 'data': 'e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0='}, '_type': 'my_type', '_version': 1, 'found': True} We can see that the binary data passed to the pipeline was a Rich Text Format file and that the content was extracted: Lorem ipsum dolor sit amet Displaying the binary data is not very useful. It doesn't matter in this example as it's quite small. But it would be much bigger even on small files. We can exclude it using _source_exclude: In [10]: es.get(index='my_index', doc_type='my_type', id=result1['_id'], _source_exclude=['data'])  Out[10]: {'_id': 'AVhvJKzVIvjFWZACJU_t', '_index': 'my_index', '_source': {'attachment': {'content': 'Lorem ipsum dolor sit amet', 'content_length': 28, 'content_type': 'application/rtf', 'language': 'ro'}}, '_type': 'my_type', '_version': 1, 'found': True} ### Indexing PDF files¶ Let's try to parse the same sample pdf as before. In [11]: url = 'http://www.cbu.edu.zm/downloads/pdf-sample.pdf' response = requests.get(url)  Note that we have to encode the content of the pdf before to pass it to ES. The source field must be a base64 encoded binary. In [12]: import base64 data = base64.b64encode(response.content).decode('ascii')  In [13]: result2 = es.index(index='my_index', doc_type='my_type', pipeline='attachment', body={'data': data}) result2  Out[13]: {'_id': 'AVhvJMC6IvjFWZACJU_u', '_index': 'my_index', '_shards': {'failed': 0, 'successful': 1, 'total': 2}, '_type': 'my_type', '_version': 1, 'created': True, 'result': 'created'} We can get the document based on its id: In [14]: doc = es.get(index='my_index', doc_type='my_type', id=result2['_id'], _source_exclude=['data']) doc  Out[14]: {'_id': 'AVhvJMC6IvjFWZACJU_u', '_index': 'my_index', '_source': {'attachment': {'author': 'cdaily', 'content': "Adobe Acrobat PDF Files\n\nAdobe® Portable Document Format (PDF) is a universal file format that preserves all\nof the fonts, formatting, colours and graphics of any source document, regardless of\nthe application and platform used to create it.\n\nAdobe PDF is an ideal format for electronic document distribution as it overcomes the\nproblems commonly encountered with electronic file sharing.\n\n• Anyone, anywhere can open a PDF file. All you need is the free Adobe Acrobat\nReader. Recipients of other file formats sometimes can't open files because they\ndon't have the applications used to create the documents.\n\n• PDF files always print correctly on any printing device.\n\n• PDF files always display exactly as created, regardless of fonts, software, and\noperating systems. Fonts, and graphics are not lost due to platform, software, and\nversion incompatibilities.\n\n• The free Acrobat Reader is easy to download and can be freely distributed by\nanyone.\n\n• Compact PDF files are smaller than their source files and download a\npage at a time for fast display on the Web.", 'content_length': 1073, 'content_type': 'application/pdf', 'date': '2000-06-28T23:21:08Z', 'language': 'en', 'title': 'This is a test PDF file'}}, '_type': 'my_type', '_version': 1, 'found': True} Or with a basic search: In [15]: es.search(index='my_index', doc_type='my_type', q='Adobe', _source_exclude=['data'])  Out[15]: {'_shards': {'failed': 0, 'successful': 5, 'total': 5}, 'hits': {'hits': [{'_id': 'AVhvJMC6IvjFWZACJU_u', '_index': 'my_index', '_score': 0.45930308, '_source': {'attachment': {'author': 'cdaily', 'content': "Adobe Acrobat PDF Files\n\nAdobe® Portable Document Format (PDF) is a universal file format that preserves all\nof the fonts, formatting, colours and graphics of any source document, regardless of\nthe application and platform used to create it.\n\nAdobe PDF is an ideal format for electronic document distribution as it overcomes the\nproblems commonly encountered with electronic file sharing.\n\n• Anyone, anywhere can open a PDF file. All you need is the free Adobe Acrobat\nReader. Recipients of other file formats sometimes can't open files because they\ndon't have the applications used to create the documents.\n\n• PDF files always print correctly on any printing device.\n\n• PDF files always display exactly as created, regardless of fonts, software, and\noperating systems. Fonts, and graphics are not lost due to platform, software, and\nversion incompatibilities.\n\n• The free Acrobat Reader is easy to download and can be freely distributed by\nanyone.\n\n• Compact PDF files are smaller than their source files and download a\npage at a time for fast display on the Web.", 'content_length': 1073, 'content_type': 'application/pdf', 'date': '2000-06-28T23:21:08Z', 'language': 'en', 'title': 'This is a test PDF file'}}, '_type': 'my_type'}], 'max_score': 0.45930308, 'total': 1}, 'timed_out': False, 'took': 75} Of course Elasticsearch allows much more complex queries. But that's something for another time. One interesting thing is that by printing the content, we can see that even the layout is quite acurate! Much better than the pdftotext output: In [16]: print(doc['_source']['attachment']['content'])  Adobe Acrobat PDF Files Adobe® Portable Document Format (PDF) is a universal file format that preserves all of the fonts, formatting, colours and graphics of any source document, regardless of the application and platform used to create it. Adobe PDF is an ideal format for electronic document distribution as it overcomes the problems commonly encountered with electronic file sharing. • Anyone, anywhere can open a PDF file. All you need is the free Adobe Acrobat Reader. Recipients of other file formats sometimes can't open files because they don't have the applications used to create the documents. • PDF files always print correctly on any printing device. • PDF files always display exactly as created, regardless of fonts, software, and operating systems. Fonts, and graphics are not lost due to platform, software, and version incompatibilities. • The free Acrobat Reader is easy to download and can be freely distributed by anyone. • Compact PDF files are smaller than their source files and download a page at a time for fast display on the Web.  The ingest-attachment plugin uses the Apache text extraction library Tika. It's really powerful. It detects and extracts metadata and text from many file types. Sending the file directly to Elasticsearch is nice, but in my use case, I'd like to process the file (change its title, move it to a specific location...) based on its content. I could of course update the document in ES after processing it. It might be better in some case to decorelate the parsing and processing from the indexing. So let's check how to use Tika from Python. ## Apache Tika¶ Tika-Python makes Apache Tika available as a Python library. It can even starts a Tika REST server in the background, but this requires Java 7+ to be installed. I prefer to run the server myself using the prebuilt docker image: docker-tikaserver. Like that I have control of what is running. $ docker run --rm -p 9998:9998 logicalspark/docker-tikaserver

We can then set Tika-Python to use Client mode only:

In [17]:
import tika
tika.TikaClientOnly = True
from tika import parser

In [18]:
parsed = parser.from_file('/tmp/pdf-sample.pdf', 'http://localhost:9998/tika')

2016-11-16 22:57:14,233 [MainThread  ] [INFO ]  Starting new HTTP connection (1): localhost

In [19]:
parsed

Out[19]:
{'content': "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThis is a test PDF file\n\n\nAdobe Acrobat PDF Files\n\nAdobe® Portable Document Format (PDF) is a universal file format that preserves all\nof the fonts, formatting, colours and graphics of any source document, regardless of\nthe application and platform used to create it.\n\nAdobe PDF is an ideal format for electronic document distribution as it overcomes the\nproblems commonly encountered with electronic file sharing.\n\n•  Anyone, anywhere can open a PDF file. All you need is the free Adobe Acrobat\nReader. Recipients of other file formats sometimes can't open files because they\ndon't have the applications used to create the documents.\n\n•  PDF files always print correctly on any printing device.\n\n•  PDF files always display exactly as created, regardless of fonts, software, and\noperating systems. Fonts, and graphics are not lost due to platform, software, and\nversion incompatibilities.\n\n•  The free Acrobat Reader is easy to download and can be freely distributed by\nanyone.\n\n•  Compact PDF files are smaller than their source files and download a\npage at a time for fast display on the Web.\n\n\n",
'Content-Type': 'application/pdf',
'Creation-Date': '2000-06-28T23:21:08Z',
'Last-Modified': '2013-10-28T19:24:13Z',
'Last-Save-Date': '2013-10-28T19:24:13Z',
'X-Parsed-By': ['org.apache.tika.parser.DefaultParser',
'org.apache.tika.parser.pdf.PDFParser'],
'X-TIKA:parse_time_millis': '62',
'access_permission:assemble_document': 'true',
'access_permission:can_modify': 'true',
'access_permission:can_print': 'true',
'access_permission:extract_content': 'true',
'access_permission:extract_for_accessibility': 'true',
'access_permission:fill_in_form': 'true',
'access_permission:modify_annotations': 'true',
'created': 'Wed Jun 28 23:21:08 UTC 2000',
'creator': 'cdaily',
'date': '2013-10-28T19:24:13Z',
'dc:creator': 'cdaily',
'dc:format': 'application/pdf; version=1.3',
'dc:title': 'This is a test PDF file',
'dcterms:created': '2000-06-28T23:21:08Z',
'dcterms:modified': '2013-10-28T19:24:13Z',
'meta:author': 'cdaily',
'meta:creation-date': '2000-06-28T23:21:08Z',
'meta:save-date': '2013-10-28T19:24:13Z',
'modified': '2013-10-28T19:24:13Z',
'pdf:PDFVersion': '1.3',
'pdf:docinfo:created': '2000-06-28T23:21:08Z',
'pdf:docinfo:creator': 'cdaily',
'pdf:docinfo:creator_tool': 'Microsoft Word 8.0',
'pdf:docinfo:modified': '2013-10-28T19:24:13Z',
'pdf:docinfo:producer': 'Acrobat Distiller 4.0 for Windows',
'pdf:docinfo:title': 'This is a test PDF file',
'pdf:encrypted': 'false',
'producer': 'Acrobat Distiller 4.0 for Windows',
'resourceName': 'pdf-sample.pdf',
'title': 'This is a test PDF file',
'xmp:CreatorTool': 'Microsoft Word 8.0',
'xmpMM:DocumentID': 'uuid:0805e221-80a8-459e-a522-635ed5c1e2e6',
'xmpTPg:NPages': '1'}}
In [20]:
print(parsed['content'].strip())

This is a test PDF file

Adobe® Portable Document Format (PDF) is a universal file format that preserves all
of the fonts, formatting, colours and graphics of any source document, regardless of
the application and platform used to create it.

Adobe PDF is an ideal format for electronic document distribution as it overcomes the
problems commonly encountered with electronic file sharing.

•  Anyone, anywhere can open a PDF file. All you need is the free Adobe Acrobat
Reader. Recipients of other file formats sometimes can't open files because they
don't have the applications used to create the documents.

•  PDF files always print correctly on any printing device.

•  PDF files always display exactly as created, regardless of fonts, software, and
operating systems. Fonts, and graphics are not lost due to platform, software, and
version incompatibilities.

anyone.

•  Compact PDF files are smaller than their source files and download a
page at a time for fast display on the Web.

Not sure why we get the title of the PDF inside the content. Anyway the text is extracted properly and we even get a lot of metadata:

In [21]:
parsed['metadata']

Out[21]:
{'Author': 'cdaily',
'Content-Type': 'application/pdf',
'Creation-Date': '2000-06-28T23:21:08Z',
'Last-Modified': '2013-10-28T19:24:13Z',
'Last-Save-Date': '2013-10-28T19:24:13Z',
'X-Parsed-By': ['org.apache.tika.parser.DefaultParser',
'org.apache.tika.parser.pdf.PDFParser'],
'X-TIKA:parse_time_millis': '62',
'access_permission:assemble_document': 'true',
'access_permission:can_modify': 'true',
'access_permission:can_print': 'true',
'access_permission:extract_content': 'true',
'access_permission:extract_for_accessibility': 'true',
'access_permission:fill_in_form': 'true',
'access_permission:modify_annotations': 'true',
'created': 'Wed Jun 28 23:21:08 UTC 2000',
'creator': 'cdaily',
'date': '2013-10-28T19:24:13Z',
'dc:creator': 'cdaily',
'dc:format': 'application/pdf; version=1.3',
'dc:title': 'This is a test PDF file',
'dcterms:created': '2000-06-28T23:21:08Z',
'dcterms:modified': '2013-10-28T19:24:13Z',
'meta:author': 'cdaily',
'meta:creation-date': '2000-06-28T23:21:08Z',
'meta:save-date': '2013-10-28T19:24:13Z',
'modified': '2013-10-28T19:24:13Z',
'pdf:PDFVersion': '1.3',
'pdf:docinfo:created': '2000-06-28T23:21:08Z',
'pdf:docinfo:creator': 'cdaily',
'pdf:docinfo:creator_tool': 'Microsoft Word 8.0',
'pdf:docinfo:modified': '2013-10-28T19:24:13Z',
'pdf:docinfo:producer': 'Acrobat Distiller 4.0 for Windows',
'pdf:docinfo:title': 'This is a test PDF file',
'pdf:encrypted': 'false',
'producer': 'Acrobat Distiller 4.0 for Windows',
'resourceName': 'pdf-sample.pdf',
'title': 'This is a test PDF file',
'xmp:CreatorTool': 'Microsoft Word 8.0',
'xmpMM:DocumentID': 'uuid:0805e221-80a8-459e-a522-635ed5c1e2e6',
'xmpTPg:NPages': '1'}

## Conclusion¶

We saw different methods to extract text from PDF in Python. Depending on what you want to do, one might suit you better. And this was of course not exhaustive.

If you want to index PDFs, Elasticsearch might be all you need. The ingest-attachment plugin uses Apache Tika which is very powerful.

And thanks to Tika-Python, it's very easy to use Tika directly from Python. You can let the library starts the server or use Docker to start your own.


text:
### Topic: Linux for Dummies?  (Read 9351 times) var addthis_config = {"data_track_clickback":true};

0 Members and 1 Guest are viewing this topic.

#### Hexx

• Sexy Shoeless Lyran God Of War
• Captain
• Posts: 6058
##### Linux for Dummies?
« on: September 22, 2007, 08:11:24 pm »
Hey all

So might be having to get a new hard drive (very very low speeds on mine atm)
Unfortunately looks like my XP cd's were in one of the missing boxes when I moved (figures)

Anyway, not really looking to spend big  on buying anotehr copy of XP, and I've heard that vista won't let me watch a bunch of the movies/TV I've downloaded
so I was thinking of trying Linux.

Only questions are
1)- Does Linux work for people who have no idea about computers, or is it more of a OS for people who know their stuff

2) Can I play Op with Linux?

3) How hard is it to install as the OS?
Courageously Protesting "Lyran Pelt Day"

#### Lepton

• Lt. Commander
• Posts: 1620
##### Re: Linux for Dummies?
« Reply #1 on: September 22, 2007, 09:53:43 pm »
Hey all

So might be having to get a new hard drive (very very low speeds on mine atm)
Unfortunately looks like my XP cd's were in one of the missing boxes when I moved (figures)

Anyway, not really looking to spend big  on buying anotehr copy of XP, and I've heard that vista won't let me watch a bunch of the movies/TV I've downloaded
so I was thinking of trying Linux.

Only questions are
1)- Does Linux work for people who have no idea about computers, or is it more of a OS for people who know their stuff

2) Can I play Op with Linux?

3) How hard is it to install as the OS?

1.  It certainly can be good for average folks if you go with a no-brainer distribution, like Ubuntu, Linux Mint, Linspire, etc.  I'd stay away from anything like Slackware or Debian.  The problem is if you really would like to do something extraordinary on Linux you're going to have to do a lot of things manually, but web browsing, music, word processing, etc is all pretty straight forward.

2.  Not that I know of.  You'd have to try to play it using WINE (free software) or Cedega (pay software).  Firesoul seemed to have got it working once but whenever I have attempted to, it's a no-go.

3.  Pretty easy.  Something like Ubuntu will guide you through the install, partition your hard drive for you and shrink your other partitions if you want to have both Windows and Linux on the same computer, etc, etc.  I installed Ubuntu on my laptop about a month ago and it was all smooth sailing.  It shrunk my Windows install down, installed Ubuntu, and I have encountered little to no problems with either the Windows or Linux side of things.

I'm sure others will pop their heads in here and give you recommendations, but if you are looking to play OP, and your XP disks are nowhere to be found, I think you are screwed.

System Specs:

Dell Dimension E521
AMD64x2 5000+
2G DDR2 RAM
ATI Radeon HD 4850 512MB GDDR3
250GB SATA HD

#### Nemesis

• Captain Kayn
• Global Moderator
• Commodore
• Posts: 12578
##### Re: Linux for Dummies?
« Reply #2 on: September 23, 2007, 06:27:20 pm »
1)- Does Linux work for people who have no idea about computers, or is it more of a OS for people who know their stuff

There are versions for the "expert" and for novices.  I suggest LinuxMint.

2) Can I play Op with Linux?

It is theoretically possible but I wouldn't count on it.

3) How hard is it to install as the OS?

LinuxMint installed with minimum skill required for me.
Do unto others as Frey has done unto you.
Seti Team    Free Software
I believe truth and principle do matter. If you have to sacrifice them to get the results you want, then the results aren't worth it.
FoaS_XC : "Take great pains to distinguish a criticism vs. an attack. A person reading a post should never be able to confuse the two."

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #3 on: September 26, 2007, 09:35:28 am »
HOLY CRAP! I did it! (well really the guys at Kubuntu and Wine...)

I got OP running on Linux!

I only got two screenshots so far, setup and the CD Key prompt. Not surprisingly the same kind of tricks to setting up OP under Wine on Linux are similar to the usual issues on windows.

Long story short - Kubuntu 7.04 (I like KDE better + K3b, Amorak... etc), I used the built in Adept package manager to install firefox and wine (0.9.45 I think it was) Make adjustments to wine in kubuntu's system setting applet if needed, though the default should be fine to start. First I tried Deus Ex, it ran pretty much out of the box, I was amazed. On to OP... install from CD no problem. Installed the patch,,. no problem, installed OP+4 with models, no problem... Many crashes and lockups later I find the shortcuts need to be edited with the working directory much as in windows, Tried hardware 3D with no luck, I installed DirectX 8.1NT in my Wine install (I know...), still no hardware, but ran the OP Setup utility (which needed the shortcut fix) to set it to software rendering at 800x600 and wow, it ran! I gleefully I entered my CD key and watched in awe as a D7 killed my CA in a quick skirmish.

(Oh yeah, i had to set my desktop to 16 bit color my manually editing /etc/X11/xorg.conf and restarting, as some of the wine debug output indicated an inability to change it through its directx interfaces.)

It was a litte choppy, but by computer is a little old... this would probably work better on a newer machine. I'll post the screenshots I have and try to get it running windowed for better demo screenshots.

I'll post more detailed and organised instructions later, I'd like to see this duplicated.

More soon... would be nice to see hardware 3D... I have an old card, might make a difference.
« Last Edit: September 26, 2007, 09:49:05 am by Bonk »

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #4 on: September 26, 2007, 10:10:52 am »
Some snapshots of the game running in windowed mode, though as with windows its finicky about that (mouse focus issues), especially while taking screenshots...  I think what's lagging it for me is the sound, I'll fiddle with Wine's sound settings to see if that helps (using OSS currently), and back to full screen..

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #5 on: September 26, 2007, 10:54:53 am »
OK, sound is definitely a problem (at least on my system) its too laggy and slow, I wonder if a faster computer would have the same issue? I'll try disabling it altogether.

Some links that helped me get this far:
http://frankscorner.org/index.php
http://www.winehq.org/

I also installed EzINI and it seems to work fine with wine, it even created a desktop shortcut! (Deus Ex and OP shortcuts came up in the lost and found - I just made some folders and desktop shortcuts for them)

Now the big question: networking?
EDIT: Doesn't look good for networking right now...  poop... perhaps a little research will help.
« Last Edit: September 26, 2007, 11:09:39 am by Bonk »

#### Hexx

• Sexy Shoeless Lyran God Of War
• Captain
• Posts: 6058
##### Re: Linux for Dummies?
« Reply #6 on: September 26, 2007, 11:16:31 am »
Cool... Thanks Bonk!

<ponders if Bonk noticed the "for dummies" >
Courageously Protesting "Lyran Pelt Day"

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #7 on: September 26, 2007, 11:54:34 am »
Cool... Thanks Bonk!

<ponders if Bonk noticed the "for dummies" >

Lol, I knew you'd start with that. I did mean to note that this first attempt is not for the novice. However I might be able to make up a set of pretty clear and reliable instructions eventually.

I just got a serverlist by following the instructions here:
http://wiki.winehq.org/DirectPlayGames
http://appdb.winehq.org/objectManager.php?sClass=version&iId=7029&iTestingId=10927

Now, I just need to solve the sound issue and test a multiplayer game!

#### Dash Jones

• Sub-Commander of the Dark Side
• Captain
• Posts: 6477
• Gender:
##### Re: Linux for Dummies?
« Reply #8 on: September 26, 2007, 04:24:13 pm »
It would be great if you would write up an article of how to do that step by step for us simpletons and post it on a site somewhere semi-permanent to permanent!
"All hominins are hominids, but not all hominids are hominins."

"Is this a Christian perspective?

Now where in the Bible does it say if someone does something stupid you should shoot them in the face?"

-------

We have whale farms in Jersey.   They're called McDonald's.

There is no "I" in team. There are two "I"s in Vin Diesel. screw you, team.

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #9 on: September 26, 2007, 11:03:23 pm »
Indeed, that was my intention. I'll keep trying to get the sound lag sorted (though that may just take a faster computer), and run a  few more tests, then write an initial guide in one of the "How To" forums.

#### Commander Maxillius

• You did NOT just shoot that green sh-t at me?!?
• Lt. Commander
• Posts: 2299
• Gender:
##### Re: Linux for Dummies?
« Reply #10 on: September 26, 2007, 11:16:19 pm »
I'm running Mint (ubuntu clone) and got OP to load in Wine, but missions wouldn't run.  I you could post the specs of your machine and (when you get the instructions hammered out) how-tos up, I'd be glad to run it up the pole.

my specs:
Athlon 2400+ (2.0 GHz)
512MB RAM (64MB taken for video)
I was never here, you were never here, this conversation never took place, and you most certainly did not see me.

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #11 on: September 27, 2007, 12:41:01 am »
Cool, I'd like to see this duplicated.

When you say missions wouldn't run do you mean even skirmish missions? Or just single player campaign missions? Or just D2 missions? Or just any missons?

I'll try a single player campaign mission and see if it will launch, though I wanted to get the missions for The Forge and test a D2 login as well. So far I have only tested single player skirmish.

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #12 on: September 27, 2007, 12:58:35 am »
* Bonk ...wonders if the serverkit will run under wine?...

#### Bonk

• Commodore
• Posts: 13298
• You don't have to live like a refugee.
##### Re: Linux for Dummies?
« Reply #13 on: September 27, 2007, 11:27:54 am »
I minimized the sound lag by selecting the ALSA driveer under wine and setting the hardware to emulation at 22kHz,8bit, though it is still a problem, it might just be that my old PC does not quite have the guts to do this.

I played some singleplayer campaign missions no problems and logged in to the D2 and multiplayer worked surprisingly well, I crashed out of one mission taking desktop screenshots (old hardware again) and managed to finish a second mission but lost connection at the end of the mission... very close. I'll have to test this some more, but I suppose a newer and faster computer on high speed may not see these issues.

#### Hexx

• Sexy Shoeless Lyran God Of War
• Captain
• Posts: 6058
##### Re: Linux for Dummies?
« Reply #14 on: September 27, 2007, 11:32:26 am »
Sweet! Grats Bonk.

<still has no idea what Bonk's talking about but assumes it's important    >
Courageously Protesting "Lyran Pelt Day"

#### FCM_SFHQ_XC

• There is life outside of Windows..
• Lt. Commander
• Posts: 2267
• Gender:
• Starbase Atlantis [X-refit]
##### Re: Linux for Dummies?
« Reply #15 on: September 27, 2007, 11:42:04 am »
<still has no idea what Bonk's talking about but assumes it's important >
Oh this is important and great news alright

Fleet Commodore, XenoCorp, ISC Fleet.

#### animator

• Ensign
• Posts: 3
##### Re: Linux for Dummies?
« Reply #16 on: September 27, 2007, 04:26:03 pm »
OMG if this  work i can give up windows almost entirely

#### Lepton

• Lt. Commander
• Posts: 1620
##### Re: Linux for Dummies?
« Reply #17 on: September 30, 2007, 05:27:45 pm »
I've gotten the single player to work as well.  Definitely need desktop color depth set at 16 bit.  Need to set display driver with  SFCSetupMFC to Software. I used ALSA and had it set to full hardware acceleration, which resulted in very crackley sound production and a delay in playing all sound effects which is not too horrible but is able to be tolerated.  This was done under Ubuntu 7.04, Wine 0.9.33.  These are much the same problems that I had when I tried to run OP on my old 500 Mhz PIII machine, except video was intermittent at best and sounds were very delayed.  This is now a AMD64 3000+, so things are certainly better, but not as good as I would like.  I am going to try it with esound instead of alsa and see if that makes any difference.  Thanks for looking into this again, Bonk.  I had pretty much given up on OP uinder linux but it is looking pretty good from here.

System Specs:

Dell Dimension E521
AMD64x2 5000+
2G DDR2 RAM
ATI Radeon HD 4850 512MB GDDR3
250GB SATA HD

#### Lepton

• Lt. Commander
• Posts: 1620
##### Re: Linux for Dummies?
« Reply #18 on: September 30, 2007, 05:33:03 pm »
This sound card's driver does not support direct access
The (slower) DirectSound HEL mode will be used instead.
This sound card's driver does not support direct access
The (slower) DirectSound HEL mode will be used instead.

This was the output I got when switching to esound.  Sound playback was entirely not functional.  At present stick with alsa.

System Specs:

Dell Dimension E521
AMD64x2 5000+
2G DDR2 RAM
ATI Radeon HD 4850 512MB GDDR3
250GB SATA HD

#### Lepton

• Lt. Commander
• Posts: 1620
##### Re: Linux for Dummies?
« Reply #19 on: September 30, 2007, 06:53:15 pm »
Online.  This is pretty cool.

System Specs:

Dell Dimension E521
AMD64x2 5000+
2G DDR2 RAM
ATI Radeon HD 4850 512MB GDDR3
250GB SATA HD

format: plain


text:
Paul's Online Notes
Home / Calculus II / Integration Techniques / Integrals Involving Trig Functions
Show Mobile Notice Show All Notes Hide All Notes
Mobile Notice
You appear to be on a device with a "narrow" screen width (i.e. you are probably on a mobile phone). Due to the nature of the mathematics on this site it is best views in landscape mode. If your device is not in landscape mode many of the equations will run off the side of your device (should be able to scroll to see them) and some of the menu items will be cut off due to the narrow screen width.

### Section 7.2 : Integrals Involving Trig Functions

In this section we are going to look at quite a few integrals involving trig functions and some of the techniques we can use to help us evaluate them. Let’s start off with an integral that we should already be able to do.

\begin{align*}\int{{\cos x\sin ^5 x\,dx}} & = \int{{{u^5}\,du}}\hspace{0.25in}{\mbox{using the substitution }}u = \sin x\\ & = \frac{1}{6}{\sin ^6}x + c\end{align*}

This integral is easy to do with a substitution because the presence of the cosine, however, what about the following integral.

Example 1 Evaluate the following integral. $\int{{{{\sin }^5}x\,dx}}$
Show Solution

This integral no longer has the cosine in it that would allow us to use the substitution that we used above. Therefore, that substitution won’t work and we are going to have to find another way of doing this integral.

Let’s first notice that we could write the integral as follows,

$\int{{{{\sin }^5}x\,dx}} = \int{{{{\sin }^4}x\,\sin x\,dx}} = \int{{{{\left( {{{\sin }^2}x} \right)}^2}\,\sin x\,dx}}$

Now recall the trig identity,

${\cos ^2}x + {\sin ^2}x = 1\hspace{0.25in}\,\, \Rightarrow \hspace{0.25in}\,\,\,{\sin ^2}x = 1 - {\cos ^2}x$

With this identity the integral can be written as,

$\int{{{{\sin }^5}x\,dx}} = \int{{{{\left( {1 - {{\cos }^2}x} \right)}^2}\,\sin x\,dx}}$

and we can now use the substitution $$u = \cos x$$. Doing this gives us,

\begin{align*}\int{{{{\sin }^5}x\,dx}} & = - \int{{{{\left( {1 - {u^2}} \right)}^2}\,du}}\\ & = - \int{{1 - 2{u^2} + {u^4}\,du}}\\ & = - \left( {u - \frac{2}{3}{u^3} + \frac{1}{5}{u^5}} \right) + c\\ & = - \cos x + \frac{2}{3}{\cos ^3}x - \frac{1}{5}{\cos ^5}x + c\end{align*}

So, with a little rewriting on the integrand we were able to reduce this to a fairly simple substitution.

Notice that we were able to do the rewrite that we did in the previous example because the exponent on the sine was odd. In these cases all that we need to do is strip out one of the sines. The exponent on the remaining sines will then be even and we can easily convert the remaining sines to cosines using the identity,

$$${\cos ^2}x + {\sin ^2}x = 1 \label{eq:eq1}$$$

If the exponent on the sines had been even this would have been difficult to do. We could strip out a sine, but the remaining sines would then have an odd exponent and while we could convert them to cosines the resulting integral would often be even more difficult than the original integral in most cases.

Let’s take a look at another example.

Example 2 Evaluate the following integral. $\int{{\sin^6 x \cos^3 x\,dx}}$
Show Solution

So, in this case we’ve got both sines and cosines in the problem and in this case the exponent on the sine is even while the exponent on the cosine is odd. So, we can use a similar technique in this integral. This time we’ll strip out a cosine and convert the rest to sines.

\begin{align*}\int{{\sin^6 x \cos^3 x\,dx}} & = \int{{\sin^6 x \cos^2 x\,\cos x\,dx}}\\ & = \int{{{{\sin }^6}x\left( {1 - {{\sin }^2}x} \right)\,\cos x\,dx}}\hspace{0.5in}u = \sin x\\ & = \int{{{u^6}\left( {1 - {u^2}} \right)\,du}}\\ & = \int{{{u^6} - {u^8}\,du}}\\ & = \frac{1}{7}{\sin ^7}x - \frac{1}{9}{\sin ^9}x + c\end{align*}

At this point let’s pause for a second to summarize what we’ve learned so far about integrating powers of sine and cosine.

$$$\int{{\sin^n x \cos^m x\,dx}} \label{eq:eq2}$$$

In this integral if the exponent on the sines ($$n$$) is odd we can strip out one sine, convert the rest to cosines using $$\eqref{eq:eq1}$$ and then use the substitution $$u = \cos x$$. Likewise, if the exponent on the cosines ($$m$$) is odd we can strip out one cosine and convert the rest to sines and the use the substitution $$u = \sin x$$.

Of course, if both exponents are odd then we can use either method. However, in these cases it’s usually easier to convert the term with the smaller exponent.

The one case we haven’t looked at is what happens if both of the exponents are even? In this case the technique we used in the first couple of examples simply won’t work and in fact there really isn’t any one set method for doing these integrals. Each integral is different and in some cases there will be more than one way to do the integral.

With that being said most, if not all, of integrals involving products of sines and cosines in which both exponents are even can be done using one or more of the following formulas to rewrite the integrand.

\begin{align*}{\cos ^2}x & = \frac{1}{2}\left( {1 + \cos \left( {2x} \right)} \right)\\ {\sin ^2}x & = \frac{1}{2}\left( {1 - \cos \left( {2x} \right)} \right)\\ \sin x\cos x & = \frac{1}{2}\sin \left( {2x} \right)\end{align*}

The first two formulas are the standard half angle formula from a trig class written in a form that will be more convenient for us to use. The last is the standard double angle formula for sine, again with a small rewrite.

Let’s take a look at an example.

Example 3 Evaluate the following integral. $\int{{\sin ^2 x \cos ^2 x\,dx}}$
Show Solution

As noted above there are often more than one way to do integrals in which both of the exponents are even. This integral is an example of that. There are at least two solution techniques for this problem. We will do both solutions starting with what is probably the longer of the two, but it’s also the one that many people see first.

Solution 1
In this solution we will use the two half angle formulas above and just substitute them into the integral.

\begin{align*}\int{{\sin ^2 x \cos ^2 x\,dx}} & = \int{{\frac{1}{2}\left( {1 - \cos \left( {2x} \right)} \right)\left( {\frac{1}{2}} \right)\left( {1 + \cos \left( {2x} \right)} \right)\,dx}}\\ & = \frac{1}{4}\int{{1 - {{\cos }^2}\left( {2x} \right)\,dx}}\end{align*}

So, we still have an integral that can’t be completely done, however notice that we have managed to reduce the integral down to just one term causing problems (a cosine with an even power) rather than two terms causing problems.

In fact to eliminate the remaining problem term all that we need to do is reuse the first half angle formula given above.

\begin{align*}\int{{\sin ^2 x \cos ^2 x\,dx}} & = \frac{1}{4}\int{{1 - \frac{1}{2}\left( {1 + \cos \left( {4x} \right)} \right)\,dx}}\\ & = \frac{1}{4}\int{{\frac{1}{2} - \frac{1}{2}\cos \left( {4x} \right)\,dx}}\\ & = \frac{1}{4}\left( {\frac{1}{2}x - \frac{1}{8}\sin \left( {4x} \right)} \right) + c\\ & = \frac{1}{8}x - \frac{1}{{32}}\sin \left( {4x} \right) + c\end{align*}

So, this solution required a total of three trig identities to complete.

Solution 2
In this solution we will use the double angle formula to help simplify the integral as follows.

\begin{align*}\int{{\sin ^2 x \cos ^2 x\,dx}} & = \int{{{{\left( {\sin x\cos x} \right)}^2}\,dx}}\\ & = \int{{{{\left( {\frac{1}{2}\sin \left( {2x} \right)} \right)}^2}\,dx}}\\ & = \frac{1}{4}\int{{{{\sin }^2}\left( {2x} \right)\,dx}}\end{align*}

Now, we use the half angle formula for sine to reduce to an integral that we can do.

\begin{align*}\int{{\sin ^2 x \cos ^2 x\,dx}} & = \frac{1}{8}\int{{1 - \cos \left( {4x} \right)\,dx}}\\ & = \frac{1}{8}x - \frac{1}{{32}}\sin \left( {4x} \right) + c\end{align*}

This method required only two trig identities to complete.

Notice that the difference between these two methods is more one of “messiness”. The second method is not appreciably easier (other than needing one less trig identity) it is just not as messy and that will often translate into an “easier” process.

In the previous example we saw two different solution methods that gave the same answer. Note that this will not always happen. In fact, more often than not we will get different answers. However, as we discussed in the Integration by Parts section, the two answers will differ by no more than a constant.

In general, when we have products of sines and cosines in which both exponents are even we will need to use a series of half angle and/or double angle formulas to reduce the integral into a form that we can integrate. Also, the larger the exponents the more we’ll need to use these formulas and hence the messier the problem.

Sometimes in the process of reducing integrals in which both exponents are even we will run across products of sine and cosine in which the arguments are different. These will require one of the following formulas to reduce the products to integrals that we can do.

\begin{align*}\sin \alpha \cos \beta & = \frac{1}{2}\left[ {\sin \left( {\alpha - \beta } \right) + \sin \left( {\alpha + \beta } \right)} \right]\\ \sin \alpha \sin \beta & = \frac{1}{2}\left[ {\cos \left( {\alpha - \beta } \right) - \cos \left( {\alpha + \beta } \right)} \right]\\ \cos \alpha \cos \beta & = \frac{1}{2}\left[ {\cos \left( {\alpha - \beta } \right) + \cos \left( {\alpha + \beta } \right)} \right]\end{align*}

Let’s take a look at an example of one of these kinds of integrals.

Example 4 Evaluate the following integral. $\int{{\cos \left( {15x} \right)\cos \left( {4x} \right)\,dx}}$
Show Solution

This integral requires the last formula listed above.

\begin{align*}\int{{\cos \left( {15x} \right)\cos \left( {4x} \right)\,dx}} & = \frac{1}{2}\int{{\cos \left( {11x} \right) + \cos \left( {19x} \right)\,dx}}\\ & = \frac{1}{2}\left( {\frac{1}{{11}}\sin \left( {11x} \right) + \frac{1}{{19}}\sin \left( {19x} \right)} \right) + c\end{align*}

Okay, at this point we’ve covered pretty much all the possible cases involving products of sines and cosines. It’s now time to look at integrals that involve products of secants and tangents.

This time, let’s do a little analysis of the possibilities before we just jump into examples. The general integral will be,

$$$\int{{ \sec^n x \tan^m x\,dx}} \label{eq:eq3}$$$

The first thing to notice is that we can easily convert even powers of secants to tangents and even powers of tangents to secants by using a formula similar to $$\eqref{eq:eq1}$$. In fact, the formula can be derived from $$\eqref{eq:eq1}$$ so let’s do that.

\begin{align}{\sin ^2}x + {\cos ^2}x & = 1 \nonumber\\ \frac{{{{\sin }^2}x}}{{{{\cos }^2}x}} + \frac{{{{\cos }^2}x}}{{{{\cos }^2}x}} & = \frac{1}{{{{\cos }^2}x}} \nonumber \\ {\tan ^2}x + 1 & = {\sec ^2}x \label{eq:eq4} \end{align}

Now, we’re going to want to deal with $$\eqref{eq:eq3}$$ similarly to how we dealt with $$\eqref{eq:eq2}$$. We’ll want to eventually use one of the following substitutions.

\begin{align*}u & = \tan x & \hspace{0.5in}du & = {\sec ^2}x\,dx\\ u & = \sec x & \hspace{0.5in}du & = \sec x\tan x\,dx\end{align*}

So, if we use the substitution $$u = \tan x$$ we will need two secants left for the substitution to work. This means that if the exponent on the secant ($$n$$) is even we can strip two out and then convert the remaining secants to tangents using $$\eqref{eq:eq4}$$.

Next, if we want to use the substitution $$u = \sec x$$ we will need one secant and one tangent left over in order to use the substitution. This means that if the exponent on the tangent ($$m$$) is odd and we have at least one secant in the integrand we can strip out one of the tangents along with one of the secants of course. The tangent will then have an even exponent and so we can use $$\eqref{eq:eq4}$$ to convert the rest of the tangents to secants. Note that this method does require that we have at least one secant in the integral as well. If there aren’t any secants then we’ll need to do something different.

If the exponent on the secant is even and the exponent on the tangent is odd then we can use either case. Again, it will be easier to convert the term with the smallest exponent.

Let’s take a look at a couple of examples.

Example 5 Evaluate the following integral. $\int{{ \sec^9 x \tan^5 x \,dx}}$
Show Solution

First note that since the exponent on the secant isn’t even we can’t use the substitution $$u = \tan x$$. However, the exponent on the tangent is odd and we’ve got a secant in the integral and so we will be able to use the substitution $$u = \sec x$$. This means stripping out a single tangent (along with a secant) and converting the remaining tangents to secants using $$\eqref{eq:eq4}$$.

Here’s the work for this integral.

\begin{align*}\int{{ \sec^9 x \tan^5 x \,dx}} & = \int{{ \sec^8 x \tan^4 x\,\tan x\sec x\,dx}}\\ & = \int{{{{\sec }^8}x{{\left( {{{\sec }^2}x - 1} \right)}^2}\,\tan x\sec x\,dx}}\hspace{0.5in}u = \sec x\\ & = \int{{{u^8}{{\left( {{u^2} - 1} \right)}^2}\,du}}\\ & = \int{{{u^{12}} - 2{u^{10}} + {u^8}\,du}}\\ & = \frac{1}{{13}}{\sec ^{13}}x - \frac{2}{{11}}{\sec ^{11}}x + \frac{1}{9}{\sec ^9}x + c\end{align*}
Example 6 Evaluate the following integral. $\int{{\sec^4 x \tan^6 x \,dx}}$
Show Solution

So, in this example the exponent on the tangent is even so the substitution $$u = \sec x$$ won’t work. The exponent on the secant is even and so we can use the substitution $$u = \tan x$$ for this integral. That means that we need to strip out two secants and convert the rest to tangents. Here is the work for this integral.

\begin{align*}\int{{\sec^4 x \tan^6 x\,dx}} & = \int{{\sec^2 x \tan^6 x\,{{\sec }^2}x\,dx}}\\ & = \int{{\left( {{{\tan }^2}x + 1} \right){{\tan }^6}x\,{{\sec }^2}x\,dx}}\hspace{0.5in}u = \tan x\\ & = \int{{\left( {{u^2} + 1} \right){u^6}\,du}}\\ & = \int{{{u^8} + {u^6}\,du}}\\ & = \frac{1}{9}{\tan ^9}x + \frac{1}{7}{\tan ^7}x + c\end{align*}

Both of the previous examples fit very nicely into the patterns discussed above and so were not all that difficult to work. However, there are a couple of exceptions to the patterns above and in these cases there is no single method that will work for every problem. Each integral will be different and may require different solution methods in order to evaluate the integral.

Let’s first take a look at a couple of integrals that have odd exponents on the tangents, but no secants. In these cases we can’t use the substitution $$u = \sec x$$since it requires there to be at least one secant in the integral.

Example 7 Evaluate the following integral. $\int{{\tan x\,dx}}$
Show Solution

To do this integral all we need to do is recall the definition of tangent in terms of sine and cosine and then this integral is nothing more than a Calculus I substitution.

\begin{align*}\int{{\tan x\,dx}} & = \int{{\frac{{\sin x}}{{\cos x}}\,dx}}\hspace{0.5in}\hspace{0.25in}\,\,\,\,\,u = \cos x\\ & = - \int{{\frac{1}{u}\,du}}\\ & = - \ln \left| {\cos x} \right| + c\hspace{0.5in}r\ln x = \ln {x^r}\\ & = \ln {\left| {\cos x} \right|^{ - 1}} + c\\ & = \ln \left| {\sec x} \right| + c\end{align*}

Note that for many folks,

$\int{{\tan x\,dx}} = - \ln \left| {\cos x} \right| + c$

We went a step or two further with some simplification. The simplification was done solely to eliminate the minus sign that was in front of the logarithm. This does not have to be done in general, but it is always easy to lose minus signs and in this case it was easy to eliminate it without introducing any real complexity to the answer and so we did.

Example 8 Evaluate the following integral. $\int{{{{\tan }^3}x\,dx}}$
Show Solution

The trick to this one is do the following manipulation of the integrand.

\begin{align*}\int{{{{\tan }^3}x\,dx}} & = \int{{\tan x \tan^2 x\,dx}}\\ & = \int{{\tan x\left( {{{\sec }^2}x - 1} \right)\,dx}}\\ & = \int{{\tan x \sec^2 x\,dx}} - \int{{\tan x\,dx}}\end{align*}

We can now use the substitution $$u = \tan x$$ on the first integral and the results from the previous example on the second integral.

The integral is then,

$\int{{{{\tan }^3}x\,dx}} = \frac{1}{2}{\tan ^2}x - \ln \left| {\sec x} \right| + c$

Note that all odd powers of tangent (with the exception of the first power) can be integrated using the same method we used in the previous example. For instance,

$\int{{{{\tan }^5}x\,dx}} = \int{{{{\tan }^3}x\left( {{{\sec }^2}x - 1} \right)\,dx}} = \int{{\tan^3 x \sec^2 x\,dx}} - \int{{{{\tan }^3}x\,dx}}$

So, a quick substitution ($$u = \tan x$$) will give us the first integral and the second integral will always be the previous odd power.

Now let’s take a look at a couple of examples in which the exponent on the secant is odd and the exponent on the tangent is even. In these cases the substitutions used above won’t work.

It should also be noted that both of the following two integrals are integrals that we’ll be seeing on occasion in later sections of this chapter and in later chapters. Because of this it wouldn’t be a bad idea to make a note of these results so you’ll have them ready when you need them later.

Example 9 Evaluate the following integral. $\int{{\sec x\,dx}}$
Show Solution

This one isn’t too bad once you see what you’ve got to do. By itself the integral can’t be done. However, if we manipulate the integrand as follows we can do it.

\begin{align*}\int{{\sec x\,dx}} & = \int{{\frac{{\sec x\left( {\sec x + \tan x} \right)}}{{\sec x + \tan x}}\,dx}}\\ & = \int{{\frac{{{{\sec }^2}x + \tan x\sec x}}{{\sec x + \tan x}}\,dx}}\end{align*}

In this form we can do the integral using the substitution $$u = \sec x + \tan x$$. Doing this gives,

$\int{{\sec x\,dx}} = \ln \left| {\sec x + \tan x} \right| + c$

The idea used in the above example is a nice idea to keep in mind. Multiplying the numerator and denominator of a term by the same term above can, on occasion, put the integral into a form that can be integrated. Note that this method won’t always work and even when it does it won’t always be clear what you need to multiply the numerator and denominator by. However, when it does work and you can figure out what term you need it can greatly simplify the integral.

Here’s the next example.

Example 10 Evaluate the following integral. $\int{{{{\sec }^3}x\,dx}}$
Show Solution

This one is different from any of the other integrals that we’ve done in this section. The first step to doing this integral is to perform integration by parts using the following choices for $$u$$ and $$dv$$.

\begin{align*}u & = \sec x &\hspace{0.5in}dv & = {\sec ^2}x\,dx\\ du & = \sec x\tan x\,dx & \hspace{0.5in}v & = \tan x\end{align*}

Note that using integration by parts on this problem is not an obvious choice, but it does work very nicely here. After doing integration by parts we have,

$\int{{{{\sec }^3}x\,dx}} = \sec x\tan x - \int{{\sec x \tan^2 x\,dx}}$

Now the new integral also has an odd exponent on the secant and an even exponent on the tangent and so the previous examples of products of secants and tangents still won’t do us any good.

To do this integral we’ll first write the tangents in the integral in terms of secants. Again, this is not necessarily an obvious choice but it’s what we need to do in this case.

\begin{align*}\int{{{{\sec }^3}x\,dx}} & = \sec x\tan x - \int{{\sec x\left( {{{\sec }^2}x - 1} \right)\,dx}}\\ & = \sec x\tan x - \int{{{{\sec }^3}x\,dx}} + \int{{\sec x\,dx}}\end{align*}

Now, we can use the results from the previous example to do the second integral and notice that the first integral is exactly the integral we’re being asked to evaluate with a minus sign in front. So, add it to both sides to get,

$2\int{{{{\sec }^3}x\,dx}} = \sec x\tan x + \ln \left| {\sec x + \tan x} \right|$

Finally divide by two and we’re done.

$\int{{{{\sec }^3}x\,dx}} = \frac{1}{2}\left( {\sec x\tan x + \ln \left| {\sec x + \tan x} \right|} \right) + c$

Again, note that we’ve again used the idea of integrating the right side until the original integral shows up and then moving this to the left side and dividing by its coefficient to complete the evaluation. We first saw this in the Integration by Parts section and noted at the time that this was a nice technique to remember. Here is another example of this technique.

Now that we’ve looked at products of secants and tangents let’s also acknowledge that because we can relate cosecants and cotangents by

$1 + {\cot ^2}x = {\csc ^2}x$

all of the work that we did for products of secants and tangents will also work for products of cosecants and cotangents. We’ll leave it to you to verify that.

There is one final topic to be discussed in this section before moving on.

To this point we’ve looked only at products of sines and cosines and products of secants and tangents. However, the methods used to do these integrals can also be used on some quotients involving sines and cosines and quotients involving secants and tangents (and hence quotients involving cosecants and cotangents).

Let’s take a quick look at an example of this.

Example 11 Evaluate the following integral. $\int{{\frac{{{{\sin }^7}x}}{{{{\cos }^4}x}}\,dx}}$
Show Solution

If this were a product of sines and cosines we would know what to do. We would strip out a sine (since the exponent on the sine is odd) and convert the rest of the sines to cosines. The same idea will work in this case. We’ll strip out a sine from the numerator and convert the rest to cosines as follows,

\begin{align*}\int{{\frac{{{{\sin }^7}x}}{{{{\cos }^4}x}}\,dx}} & = \int{{\frac{{{{\sin }^6}x}}{{{{\cos }^4}x}}\,\sin x\,dx}}\\ & = \int{{\frac{{{{\left( {{{\sin }^2}x} \right)}^3}}}{{{{\cos }^4}x}}\,\sin x\,dx}}\\ & = \int{{\frac{{{{\left( {1 - {{\cos }^2}x} \right)}^3}}}{{{{\cos }^4}x}}\,\sin x\,dx}}\end{align*}

At this point all we need to do is use the substitution $$u = \cos x$$and we’re done.

\begin{align*}\int{{\frac{{{{\sin }^7}x}}{{{{\cos }^4}x}}\,dx}} & = - \int{{\frac{{{{\left( {1 - {u^2}} \right)}^3}}}{{{u^4}}}\,du}}\\ & = - \int{{{u^{ - 4}} - 3{u^{ - 2}} + 3 - {u^2}\,du}}\\ & = - \left( { - \frac{1}{3}\frac{1}{{{u^3}}} + 3\frac{1}{u} + 3u - \frac{1}{3}{u^3}} \right) + c\\ & = \frac{1}{{3{{\cos }^3}x}} - \frac{3}{{\cos x}} - 3\cos x + \frac{1}{3}{\cos ^3}x + c\end{align*}

So, under the right circumstances, we can use the ideas developed to help us deal with products of trig functions to deal with quotients of trig functions. The natural question then, is just what are the right circumstances?

First notice that if the quotient had been reversed as in this integral,

$\int{{\frac{{{{\cos }^4}x}}{{{{\sin }^7}x}}\,dx}}$

we wouldn’t have been able to strip out a sine.

$\int{{\frac{{{{\cos }^4}x}}{{{{\sin }^7}x}}\,dx}} = \int{{\frac{{{{\cos }^4}x}}{{{{\sin }^6}x}}\,\,\frac{1}{{\sin x}}\,\,dx}}$

In this case the “stripped out” sine remains in the denominator and it won’t do us any good for the substitution $$u = \cos x$$since this substitution requires a sine in the numerator of the quotient. Also note that, while we could convert the sines to cosines, the resulting integral would still be a fairly difficult integral.

So, we can use the methods we applied to products of trig functions to quotients of trig functions provided the term that needs parts stripped out in is the numerator of the quotient.


text:
# Provisioning storage

Determine whether the available, unused storage is sufficient and provision it. For specifics, refer to your architecture document. The required minimums are as follows:
• 50GB for /opt/zenoss_analytics. This is where the Analytics software is installed, and is also used as temporary storage by the ETL processes.
• 500GB for /var/lib/mysql. This is where mySQL stores its data for both the data analytics database and the data warehouse.
• 10GB for /tmp. This is used as temporary storage by both the Analytics ETL application and mySQL.
1. Log in to the candidate host as root or as a user with superuser privileges.
2. Display the available storage devices:
lsblk --output=NAME,SIZE
3. Compare the available storage with the amount required for an Analytics server, identify an appropriate partition, and then create the appropriate file system.
4. For each type of storage that requires provisioning, identify an appropriate partition, and create and mount the appropriate file system.
1. Identify the target primary partition for the file system to create.
lsblk --output=NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT
2. Create an EXT4 file system of the partition.
mkfs -t ext4 <partition>
3. Create the mount point for the file system.
mkdir -p <filesystem path>

For example, mkdir -p /opt/zenoss_analytics.

4. Add an entry for the file system to the /etc/fstab file. Replace <partition> with the path of the partition from step b and the <filesystem path> from that used in the previous step:
echo "<partition> <filesystem path> ext4 defaults 0 0" >> /etc/fstab
5. Mount the file system, and then verify that it mounted correctly. Replace the <file system path> with the actual path.
mount -a && mount | grep <filesystem path>


text:
# Axalto Egate Drivers V3.0.6.0 Msi

Axalto Egate Drivers V3.0.6.0 Msi

A:

How do I unzip the contents of a compressed file?

It's a Windows installer package. It's a ZIP archive containing an MSI.

Axalto egate drivers v3.0.6.0 msi

This is an MSI. It has a ZIP file inside.

axalto-egate-drivers-v3.0.6.0.msi

This is a ZIP archive containing the MSI.

Q:

Limit of trigonometric functions

How to find this limit $$\lim_{x\to0}\left(\frac{\sin x}{x}+x\cos x\right)$$
I know the answer is $$-\frac{\sin x}{x}+x\cos x$$ but how did they get there.

A:

So we have,
$$\lim_{x\to0}x\left(\frac{\sin(x)}{x}+\cos(x)\right)$$
Lets divide by $x$,
$$=\lim_{x\to0}\left(1+\cos(x)\right)$$
$$=1+\lim_{x\to0}\cos(x)$$
$$=1+0$$

A:

Note that we are told that $x$ is very close to $0$. Therefore, we can write $\sin x = x - x^3\mathcal O(x^4)$ and $\cos x = \mathcal O(x^3)$. Because $x^2 = \mathcal O(x^3)$, we can also write the limit as

\begin{align}
\lim_{x \to 0} \left(\frac{\sin x}{x} + x\cos x\right)
&= \lim_{x \to 0} \frac{\sin x}{x} + \lim_{x \to 0} x\cos x \\
&= \lim_{x \to 0} \frac{\sin x}{x} + \lim_{x \to 0} x\cos x \\
&= \lim_{x \to 0} \frac{\sin x}{x} + \lim_{x \to 0} x\cos x \\
&= \lim_{x \to 0} \frac{\mathcal O(

Axalto egate drivers v3.0.6.0 msi
Axalto egate drivers v3.0.6.0 msi
Axalto egate drivers v3.0.6.0 msi
Axalto egate drivers v3.0.6.0 msi
Axalto e-gate driver 2015 Â· axalto e-gate ââ.rar.exe 1.58MB Â· Axalto-egate-drivers-v3.0.6.0.msi. 720 KB.


text:
## Your homemade pasta sauce recipe calls for 4 ounces of olive oil. You mistakenly use 3 3/4 ounces what is the relative error?

Question

Your homemade pasta sauce recipe calls for 4 ounces of olive oil. You mistakenly use 3 3/4 ounces what is the relative error?

in progress 0
3 months 2022-02-15T17:40:59+00:00 1 Answer 0 views 0

1. The relative error is: 0.0625 or 0.0625 * 100 = 6.25%

Step-by-step explanation:

Relative error is given by the formula:

$$Relative\ error = \frac{absolute\ error}{Actual\ Value}$$

While the absolute error is given by:

$$Absolute\ Error = |Measured – Actual|$$

Given

Required Sauce = Actual = 4 ounces

Added Sauce = Measured = 3 3/4 = 15/4 ounces

So,

$$absolute\ error = |\frac{15}{4} – 4|\\= |\frac{15-16}{4}|\\=|\frac{-1}{4}|\\= \frac{1}{4} => 0.25$$

Putting the values in the formula

$$Relative\ error = \frac{0.25}{4} = 0.0625$$

Hence,

The relative error is: 0.0625 or 0.0625 * 100 = 6.25%

Keywords: Relative error, error


text:
# What is the valency of sodium phosphate?

## What is the valency of sodium phosphate?

Explanation: This is so since sodium is in group 1 so has a valency of 1+ while phosphate is a polyatomic ion of valency 3− and thus we need 3 sodium atoms for each 1 phosphate ion, hence, Na3PO4 .

How do you find the valency of sodium phosphate?

The valency of sodium is 1 and that of phosphate radical is 3. So cross-multiplying the valencies we get the formula Na3PO4.

### What is the compound for sodium phosphate?

Trisodium phosphateTrisodium phosphate / IUPAC ID

What is the formula for Na3PO4?

Na₃PO₄Trisodium phosphate / Formula

## What is the compound of H3PO4?

Phosphoric acidPhosphoric acid / IUPAC ID
Phosphoric acid (orthophosphoric acid, monophosphoric acid or phosphoric(V) acid) is a weak acid with the chemical formula H3PO4. The pure compound is a colorless solid.

What is any 3 PO 4?

Other names – Trisodium phosphate, Sodium orthophosphate, Tribasic sodium phosphate

Na3PO4 Sodium Phosphate
Molecular Weight/ Molar Mass 163.94 g/mol
Boiling Point 100 °C
Melting Point 1,583 °C
Chemical Formula Na3PO4

### What is the correct name for h2o?

Oxidane
Water
Water/IUPAC ID
Dihydrogen monoxide is a name for the water molecule, which comprises two hydrogen atoms and one oxygen atom (H2O).

What is the formula for the compound formed by ba2+ and OH?

Barium hydroxide | Ba(OH)2 – PubChem.

## What is the name for the formula Na2CO3?

Sodium carbonate | Na2CO3 – PubChem.

How do you calculate phosphate?

PO₄³⁻Phosphate / Formula

### How many sodium ions are in the formula for sodium phosphate?

Sodium Phosphate is an ionic compound formed by two ions, Sodium Na+ and Phosphate PO−34 .

What is the formula for the compound formed by Na+ and P3?

“Names and Formulas of Inorganic Compounds”

A B
formula of Al3+ and Cl- AlCl3
formula of Na+ and N3- Na3N
formula of Ca2+ and O2- CaO
formula of Ca2+ and P3- Ca3P2

## What is the valency of sodium phosphate in Na3PO4?

This is because sodium belongs to group 1 and has a valency of +1, but phosphate is a polyatomic ion with a valency of 3, requiring three sodium atoms for each phosphate ion, resulting in Na3PO4. 1. is sodium phosphate soluble?

What is the chemical name of sodium phosphate?

Sodium phosphate is a sodium-phosphate salt with the chemical formula Na3PO4. This is because sodium belongs to group 1 and has a valency of +1, but phosphate is a polyatomic ion with a valency of 3, requiring three sodium atoms for each phosphate ion, resulting in Na3PO4. 1. is sodium phosphate soluble?

### Is sodium phosphate ionic or covalent?

Sodium phosphate is an ionic compound composed of sodium cation and phosphate anion. This is salt may be found in both hydrated and anhydrous forms, with anhydrous (water-free) sodium phosphate being more prevalent in nature. The sodium phosphate formula is Na3PO4 and it has a molecular weight of 163.94 g/mol.

What is the nett charge of Na3PO4?

Explanation: This is so since sodium is in group 1 so has a valency of 1+ while phosphate is a polyatomic ion of valency 3− and thus we need 3 sodium atoms for each 1 phosphate ion, hence, Na3PO4. Overall nett charge of molecule must be zero. The ions will be held together by strong ionic bonds and the molecules by strong coulombic…


text:
# What is the meaning of a “halide component” for Friedel–Crafts reaction? [closed]

Among the given halides, which one can be used as halide component for Friedel–Crafts reaction?

(a) Isopropyl chloride
(b) Bromobenzene
(c) Chlorobenzene
(d) Chloroethene

I don't understand what a "halide component" is. Is it the reactant that we use with existing molecule (a simple benzene ring, for example) on which we want to perform FC reaction on, or the reagent itself (i.e. anhyd. $$\ce{AlCl3}$$)?

• Obviously, none of these is AlCl3. Dec 1 '21 at 4:58
• Can you please elaborate? Dec 1 '21 at 5:24

$$\ce{R-X}$$
where $$\ce{R}$$ is the alkyl group and $$\ce{X}$$ is the halide (whatever it may be), I assume the question is asking what compound, of the previously noted form, could be added to an aromatic ring using electrophilic aromatic substitution.


text:
# DAG partitioning to subgraphs

Given a DAG with $|V| = n$ and has $s$ sources, we have to present subgraphs such that each subgraph has approximately $k_1=\sqrt{s}$ sources and approximately $k_2=\sqrt{n}$ nodes.

(Note: Approximately means that each subgraph contains $\lceil \sqrt{n}\rceil$ or $\lfloor \sqrt{n} \rfloor$ nodes and covers $\lceil \sqrt{s}\rceil$ or $\lfloor \sqrt{s} \rfloor$ sourses of the original graph. All sources of the original graph have to be covered by some subgraph, so there has to be $\lceil \sqrt{s}\rceil$ or $\lfloor \sqrt{s} \rfloor$ subgraphs.)

Assume following about the graph G(V,E):

1. We try to solve the problem for graphs in which such partition exists - if the partition doesn't exist it can be stated that it is impossible to create partition
2. All the graph's node will have $\forall v \in V\$ in_deg(v)=2 or in_deg(v)=1

Let's define the height of the DAG to be the maximum path length from some source to some sink.

The subgraphs have following requirements:

1. We require that all subgraphs generated will have the same height( max length of longest path)
2. Nodes of each subgraph should be reachable from the sources within that subgraph, using nodes of that subgraph as intermediate nodes.
3. Moreover, the intersection of each pair of node sets (of subgraphs) must be empty.

In the following picture, you can see an example of a right partition (assume that each edge in the graph is directed upwards).

There are 36 nodes and 8 sources [#10,11,12,13,20,21,22,23] in the example. So each subgraph should have 6 nodes and 2 or 3 sources.

Do you have idea for algorithm?

Thank you very much

-
@Yakov: do you mean that each subgraph must be 'weakly connected' (connected if you replace directed edges with undirected edges) but does not have to be 'strongly connected' (there does not necessarily exist a directed path between any two vertices)? Also: by 'leaves' do you mean vertices with out-degree of zero (also known as 'sinks')? – Niel de Beaudrap Oct 5 '10 at 15:00
@Yakov: It is much clearer now. Just a few questions: 1) haven't you confused sources & mixes? since you said edges are directed upwards, nodes [#10,...23] must be sources rather than sinks. 2) What do you mean by "approximately"? what accuracy is required? 3) how many subgraphs should the algorithm find? Apparently, trivially outputting 1 subgraph satisfies all the requirements! – M.S. Dousti Oct 6 '10 at 18:55
this was just crossposted on MO: mathoverflow.net/questions/41376/… – Suresh Venkat Oct 7 '10 at 8:21
1. Your example has the nodes of each subgraph being reachable from the sources within that subgraph, using nodes of that subgraph as intermediate nodes. Is that a requirement? If so please edit your question to reflect that. 2. Is the height of a subgraph the max length of longest path or max length of shortest path? – Warren Schudy Oct 9 '10 at 13:34
Can you tell us why you care about this problem? That way we might answer the question you really care about rather than a long sequence of questions that are close to the one you want answered. – Warren Schudy Oct 9 '10 at 17:55


text:
# Problems from Another Time

Individual problems from throughout mathematics history, as well as articles that include problem sets for students.

Given the frustum of a circular cone with height h.
A set of n disjoint, congruent circles packs the surface of a sphere S so that each region of the surface exterior to the circles is bounded by arcs of three of the circles.
An oracle ordered a prince to build a sacred building, whose space would be 400 cubits, the length being 6 cubits more than the width, and the width 3 cubits more than the height. Find the dimensions of the building.
Discussion of 15th century French manuscript, with translation of its problems, including one with negative solutions
An official asks a woman why she has so many bowls to wash. The woman explains that she had dinner guests who ate meat, rice, and soup. Judging by the number of bowls, how many guests were there?
Now a good horse and an inferior horse set out from Chang'an to Qi. Qi is 3000 li from Chang'an.
The steamer, Katie, leaves the wharf at New Orleans and runs an average speed of 15 mph. When Katie had gone 25 miles, the steamer R.E. Lee leaves the wharf and runs the average speed of 18 mph. How far will the Lee go before she overtakes the Katie?
Two bicyclists travel in opposite directions around a quarter-mile track and meet every 22 seconds. When they travel in the same direction on this track, the faster passes the slower once every 3 minutes and 40 seconds. Find the rate of each rider
Having been given the perimeter and perpendicular of a right angled triangle, it is required to find the triangle.
A farmer sold a team of horses for $440, but did not receive his pay for them until 1 yr, 8 mo after the sale. He had at the same time another offer of$410 for them. Did he gain or lose by the sale and by how much, money being worth 6%/yr?


text:
Question #da76e

$4 x$
Remember the power rule: If $f \left(x\right) = {x}^{b}$, then $f ' \left(x\right) = b {x}^{b - 1}$
$f \left(x\right) = 2 {x}^{2}$
$f ' \left(x\right) = \left(2 {x}^{2}\right) ' = 2 \cdot 2 x = 4 x$


text:
# What is the distance between (6,5) and (52,-12)?

May 22, 2018

See a solution process below:

#### Explanation:

The formula for calculating the distance between two points is:

$d = \sqrt{{\left(\textcolor{red}{{x}_{2}} - \textcolor{b l u e}{{x}_{1}}\right)}^{2} + {\left(\textcolor{red}{{y}_{2}} - \textcolor{b l u e}{{y}_{1}}\right)}^{2}}$

Substituting the values from the points in the problem gives:

$d = \sqrt{{\left(\textcolor{red}{52} - \textcolor{b l u e}{6}\right)}^{2} + {\left(\textcolor{red}{- 12} - \textcolor{b l u e}{5}\right)}^{2}}$

$d = \sqrt{{46}^{2} + {\left(- 17\right)}^{2}}$

$d = \sqrt{2116 + 289}$

$d = \sqrt{2405}$

$d = \sqrt{2405}$

Or

$d \cong 49.04$


text:
# How to make good market/shopping/grocery list in LaTeX? [closed]

I want to make a market list where the column quantity is blank. The hard copy of the list will be used as a template for market list. Housewife/House keeper will fill the quantity in the hard copy she will need corresponding to the item.

Now, my question is that is there any good package that makes the market list good one?

MWE(compile using xelatex)

\documentclass{article}
\usepackage{booktabs}
\usepackage{polyglossia}
%\setmainlanguage[numerals=Devanagari]{bengali}
%\setmainlanguage{bengali}
%\setotherlanguage{english}
%\newfontfamily\englishfont[Scale=MatchLowercase]{Linux Biolinum O}
%\newfontfamily\bengalifont[Script=Bengali]{Akaash}
\begin{document}
\begin{tabular}{l@{\hspace{1.5cm}}}
Name\\
\toprule
Rice\\
\midrule
pulse\\
\midrule
Salt\\
\midrule
Oil\\
\midrule
Pepper\\
\midrule
Onion\\
\midrule
Garlic\\
\midrule
Ginger\\
\midrule
\midrule
Chicken\\
\midrule
Spice\\
\bottomrule
\end{tabular}
\end{document}

• @AlanMunn I changed the Bangla text to English. Mar 13 '18 at 0:30
• It's still not clear what your question is, though. Mar 13 '18 at 0:33
• @AlanMunn I try to edit again. Mar 13 '18 at 0:34
• @AlanMunn I edited again. If further editing needed tell me. Mar 13 '18 at 0:54
• My point was that your example code still has the problem @AlanMunn pointed out, since most people will still be unable to compile it. I am extremely unlikely to offer a solution given that I have no idea what the problem is. However, I'll bear that criterion in mind should my crystal ball come up with anything plausible. Right now, it's just displaying an error and asking to be rebooted, but I tried that already and it didn't fix anythng.
– cfr
Mar 13 '18 at 1:24

Something like this?

\documentclass{article}
\usepackage{multicol}
\usepackage{pgffor}
\raggedcolumns
\newcommand{\ListItem}[1]{\underline{\makebox[2cm][l]{#1}}}
\begin{document}
\begin{multicols}{4}

• So remove the multicols environment. Mar 13 '18 at 9:49


text:
# Find the determinant of the matrix $\begin{vmatrix} 2 & 7 & 37\\ 3& 6 & 33\\ 4 & 5 & 29 \end{vmatrix}$

32 views

closed
Find the determinant of the matrix $\begin{vmatrix} 2 & 7 & 37\\ 3& 6 & 33\\ 4 & 5 & 29 \end{vmatrix}$
1. 234
2. 132
3. 83
4. 0
5. None of these

by (54.3k points)
selected

Correct Answer - Option 4 : 0

Concept:

Properties of Determinant of a Matrix:

• If each entry in any row or column of a determinant is 0, then the value of the determinant is zero.
• For any square matrix say A, |A| = |AT|.
• If we interchange any two rows (columns) of a matrix, the determinant is multiplied by -1.
• If any two rows (columns) of a matrix are same then the value of the determinant is zero.

Calculation:

$\begin{vmatrix} 2 & 7 & 37\\ 3& 6 & 33\\ 4 & 5 & 29 \end{vmatrix}$

Apply C2 → 5C2 + C1, we get

$\begin{vmatrix} 2 & 37 & 37\\ 3& 33 & 33\\ 4 & 29 & 29 \end{vmatrix}$

As we can see that the second and the third column of the given matrix are equal.

We know that, if any two rows (columns) of a matrix are same then the value of the determinant is zero.

∴ $\begin{vmatrix} 2 & 7 & 37\\ 3& 6 & 33\\ 4 & 5 & 29 \end{vmatrix}$ = 0


text:
# A standard pattern for class attributes as interfaces

XMLWordPrintable

#### Details

• Type: RFC
• Resolution: Unresolved
• Component/s:
• Labels:
None

#### Description

It is a relatively common problem in Python to define a base-class interface that demands that all derived classes provide a class attribute with a particular name, e.g.:

 class BaseAlgorithm(ABC):  ConfigClass = None  """Type of the configuration object for this algorithm.    Must be set by all derived classes to a subclass of  lsst.pex.config.Config.  """

The approach above is ubiquitous in our codebase (especially in Task), but I regard it as a serious antipattern, because it is all too easy for derived classes to ignore this instruction, especially if the base class initializes the attribute to a value of the appropriate type as a sort of attempt at documentation (rather than a viable default), yielding silently incorrect behavior (I have seen this in the wild, in our codebase, with the problem going unnoticed for months).

A slightly better approach is to leave the attribute undefined in the base class, which can be done by using a type annotation instead:

 class BaseAlgorithm(ABC):  ConfigClass: typing.Type[lsst.pex.config.Config]  """Type of the configuration object for this algorithm.    Must be set by all derived classes to a subclass of  lsst.pex.config.Config.  """

This will cause any attempt to access the attribute to fail, as it should, and the type annotation clarifies the type on its own.  But it'd be nice if we could get a failure earlier.  We can move it up to the point that the derived class is constructed by using the abc module an a classmethod, instead of an attribute:

 class BaseAlgorithm(ABC):  @classmethod  @abstractmethod  def getConfigClass(cls):  """Return the type of the configuration object for this algorithm."""  raise NotImplementedError()

This is my preferred solution to the problem, because:

• It is the built-in approach to defining abstract interfaces in Python.
• It is immediately obvious to any reader familiar with the abc module (and all of our developers should be, I think) what is expected of any derived class, without any documentation.  (I am not by any means suggesting that documentation could or should be skipped in this case - but it is all too easy for the author of a class to unknowingly provide incomplete or vague documentation, especially for derived-class implementers, and in-code declaration of an abstract method is a strong guard against that.)
• The abc module provides features for edge cases like, "I want to provide a default, but want to require derived classes to opt-in" (i.e. an abstractmethod with an implementation that's called with super).
• Being able to change the implementation to calculate something rather than return a constant is more often a feature than a bug (especially because we are talking about defining an interface without assuming particular implementations).
• Static analysis tools (at least MyPy; maybe some linters, too?) are aware of the abc module and will catch cases where such a method has been forgotten but code attempts to construct the derived class instance anyway.
• I believe an instantiation-time check (when static checking is possible) is enough to catch violations early enough, in practice.
• No development or maintenance of a custom solution (which would have to involve some amount of metaprogramming) is needed.

I am deeply unsympathetic to the argument that we can't tolerate an extra pair of parenthesis to make this a method call.

In the code-review debate that inspired this RFC, Nate Lust made a point that I am much more sympathetic to: that the instantiation-time checks done by abc are both later than we'd like and incomplete (they just check that a derived class attribute exists, not the degree to which it is compatible with the base class).  I don't think this is enough of a problem that it merits ignoring the built-in solution in favor of providing our own, and I think I'd also vote for the undefined-but-type-annotated approach over any custom metaprogramming I can think of (but we'll see if we get anything more concrete).

In any case, I think we do need to write something down in the dev guide (and provide utility code, if that's what we decide to recommend), because this is a common problem, we've got (IMO) a bad solution out in the wild and likely to be cargo-culted, and we shouldn't have developers inventing their own solutions every time they come across it (especially if it involves new metaprogramming each time).  Since I wrote this RFC, I'm considering the abstract classmethod approach to be the proposal on the table, but I'm curious what others think.

#### Activity

Hide
Fred Moolekamp added a comment -

+1 for preferring the "abstractmethod" solution. I also sympathize with Nate's argument and agree that it isn't foolproof, but I think that it is both better than what we have and a good solution that I have seen used effectively in other software.

Show
Fred Moolekamp added a comment - +1 for preferring the "abstractmethod" solution. I also sympathize with Nate's argument and agree that it isn't foolproof, but I think that it is both better than what we have and a good solution that I have seen used effectively in other software.
Hide
Kian-Tat Lim added a comment -

How about using @abstractmethod along with @property to get more of the syntax of the current code with the safety of the proposal?

Show
Kian-Tat Lim added a comment - How about using @abstractmethod along with @property to get more of the syntax of the current code with the safety of the proposal?
Hide
Jonathan Sick added a comment -

I agree that abstractmethod and property are what I'd expect as a Python programmer; "get" methods aren't common for this application.

Show
Jonathan Sick added a comment - I agree that abstractmethod and property are what I'd expect as a Python programmer; "get" methods aren't common for this application.
Hide
Jim Bosch added a comment -

I should have clarified the property possibility up-front, because this isn't the first time it was brought up: property only works on instances, not types, and I'm specifically concerned here with attributes that are accessible on the type (as well as the instance). For instance attributes, I do think

 @property @abstractmethod def my_attribute(self):  raise NotImplementedError()

is a very good pattern.

There are third-party implementations of a "class property" out there (including one in astropy, IIRC), but they all have some drawbacks, which is why you won't find one in the standard library. I don't know if any of them play nicely with ABC, but I personally don't consider adopting one to be a worthwhile trade for dropping parentheses.

Show
Jim Bosch added a comment - I should have clarified the property possibility up-front, because this isn't the first time it was brought up: property only works on instances , not types , and I'm specifically concerned here with attributes that are accessible on the type (as well as the instance). For instance attributes, I do think @property @abstractmethod def my_attribute(self): raise NotImplementedError() is a very good pattern. There are third-party implementations of a "class property" out there (including one in astropy, IIRC), but they all have some drawbacks, which is why you won't find one in the standard library. I don't know if any of them play nicely with ABC, but I personally don't consider adopting one to be a worthwhile trade for dropping parentheses.
Hide
Nate Lust added a comment -

I personally think this style (abstract class method) to get a simple attribute is:

• Overly verbose and inefficient
• distracting and harder to read on what is going on, and ensure you conform to some api
• Problematic in that is is late error throwing, A problem with defining a class should raise when the class is defined

I think it is a big hole in python right now that there is no standard lib way of doing this, but I don't think there is any pressure to change it because there ARE ways to get the desirable behavior, and any project is free to use the language features to implement it. What I am proposing can be accomplished with a simple class decorator. Users would do something like:

 from lsst.utils import checkAbstractAttributes, AbstractAttribute   @checkAbstractAttributes class Base:  ATTRIBUTE = AbstractAttribute   #Attempting to do this will cause some sort of exception (Runtime or something else we choose) class Test(Base):  pass

Moreover if we chose to, we could also have it raise if an abstractmethod is present at declaration time if we chose to, thus avoiding errors that might happen in a completely different file.

Also, if it is desirable we could even make it a standalone script and put into PyPi, if we wanted it to not be locked up in the stack.

Show
Nate Lust added a comment - I personally think this style (abstract class method) to get a simple attribute is: Overly verbose and inefficient distracting and harder to read on what is going on, and ensure you conform to some api Problematic in that is is late error throwing, A problem with defining a class should raise when the class is defined I think it is a big hole in python right now that there is no standard lib way of doing this, but I don't think there is any pressure to change it because there ARE ways to get the desirable behavior, and any project is free to use the language features to implement it. What I am proposing can be accomplished with a simple class decorator. Users would do something like: from lsst.utils import checkAbstractAttributes, AbstractAttribute   @checkAbstractAttributes class Base: ATTRIBUTE = AbstractAttribute   #Attempting to do this will cause some sort of exception (Runtime or something else we choose) class Test(Base): pass Moreover if we chose to, we could also have it raise if an abstractmethod is present at declaration time if we chose to, thus avoiding errors that might happen in a completely different file. Also, if it is desirable we could even make it a standalone script and put into PyPi, if we wanted it to not be locked up in the stack.
Hide
John Parejko added a comment -

I too was going to suggest an abstract method and a property, but as you say that doesn't work on class attributes. Having a get for a single value like this seems odd and unnatural (in python!) to me. Sadly, dataclasses won't work for this either. All of the other approaches I've done that I can think of also involved instance attributes.

I do agree that adding a "use property+abstractmethod for instance attributes in abstract base classes" is worth adding to the dev guide, independent of the rest of this RFC.

One advantage to the get member style proposed here is that the documentation for that attribute does get included in the output docs, whereas it doesn't for the typical "write a docstring under the attribute declaration" style, which raises this approach a little bit in my view.

What are the drawbacks of the astropy "class property"? I wasn't aware of it.

Nate's proposal requires us to write and support the code he describes; if that's more than an hour or two of work, I'm not sure it's worth while. Although, if it is good and we can get it lifted into pip, maybe that's a service. Are there any similar things already broadly available?

Show
John Parejko added a comment - I too was going to suggest an abstract method and a property, but as you say that doesn't work on class attributes. Having a get for a single value like this seems odd and unnatural (in python!) to me. Sadly, dataclasses won't work for this either. All of the other approaches I've done that I can think of also involved instance attributes. I do agree that adding a "use property+abstractmethod for instance attributes in abstract base classes" is worth adding to the dev guide, independent of the rest of this RFC. One advantage to the get member style proposed here is that the documentation for that attribute does get included in the output docs, whereas it doesn't for the typical "write a docstring under the attribute declaration" style, which raises this approach a little bit in my view. What are the drawbacks of the astropy "class property"? I wasn't aware of it. Nate's proposal requires us to write and support the code he describes; if that's more than an hour or two of work, I'm not sure it's worth while. Although, if it is good and we can get it lifted into pip, maybe that's a service. Are there any similar things already broadly available?
Hide
Nate Lust added a comment -

I can do a search to see if there is anything more broadly available, but I already wrote the other code, its about 15 lines, and took 5-10 minutes.

Show
Nate Lust added a comment - I can do a search to see if there is anything more broadly available, but I already wrote the other code, its about 15 lines, and took 5-10 minutes.
Hide
Jim Bosch added a comment -

What are the drawbacks of the astropy "class property"? I wasn't aware of it.

I can't speak to astropy's implementation specifically; the general problem is that if you define a descriptor that implements dunder-get (not gonna fight your underscore-handling today, Jira!) to return something other than self when it is given the type, rather than an instance, it's really hard to get access to the descriptor object anymore, and that plays havoc with at least docs:

 class ExampleClass:  @some_class_property  def my_class_property(cls):  return 4   print(ExampleClass.my_class_property.__doc__) # docs for int!

I don't recall if it's worse than that, and it's even possible that it's actually better than that, because there might be tools clever enough to do something like

 ExampleClass.__dict__["my_class_property"].__docs__

to grab docs or do other things with the descriptor itself; this is all just from my memories of looking around a while back for solutions and finding that the language boxes you in pretty thoroughly, and thinking that it wasn't worth it for just some mild syntactic sugar.

Which, in the end, is how I feel about metaprogramming solutions to this RFC's more immediate problem as well, especially because the 5-10 minutes it takes to code up the first draft of a bit of metaprogramming is usually a small fraction of the total cost of ownership for such things.

Show
Jim Bosch added a comment - What are the drawbacks of the astropy "class property"? I wasn't aware of it. I can't speak to astropy's implementation specifically; the general problem is that if you define a descriptor that implements dunder-get (not gonna fight your underscore-handling today, Jira!) to return something other than self when it is given the type, rather than an instance, it's really hard to get access to the descriptor object anymore, and that plays havoc with at least docs: class ExampleClass: @some_class_property def my_class_property(cls): return 4   print(ExampleClass.my_class_property.__doc__) # docs for int ! I don't recall if it's worse than that, and it's even possible that it's actually better than that, because there might be tools clever enough to do something like  ExampleClass.__dict__[ "my_class_property" ].__docs__ to grab docs or do other things with the descriptor itself; this is all just from my memories of looking around a while back for solutions and finding that the language boxes you in pretty thoroughly, and thinking that it wasn't worth it for just some mild syntactic sugar. Which, in the end, is how I feel about metaprogramming solutions to this RFC's more immediate problem as well, especially because the 5-10 minutes it takes to code up the first draft of a bit of metaprogramming is usually a small fraction of the total cost of ownership for such things.
Hide
Kian-Tat Lim added a comment -

Isn't raising an exception for a class definition actually a problem because it doesn't allow progressive concretizing in a subclass hierarchy but instead forces only concrete subclasses immediately?

 class Abstract(ABC):  @abstractmethod  @classmethod  def foo(cls):  raise NotImplementedError()    @abstractmethod  def bar(self):  raise NotImplementedError()   class LessAbstract(Abstract):  @classmethod  def foo(cls):  return 1   class Concrete(LessAbstract):  def bar(self):  return 2

I would argue that instantiation is exactly when a concreteness failure should be recognized – although that could be via static checker and not at runtime.

Show
Kian-Tat Lim added a comment - Isn't raising an exception for a class definition actually a problem because it doesn't allow progressive concretizing in a subclass hierarchy but instead forces only concrete subclasses immediately? class Abstract(ABC): @abstractmethod @classmethod def foo(cls): raise NotImplementedError() @abstractmethod def bar(self): raise NotImplementedError()   class LessAbstract(Abstract): @classmethod def foo(cls): return 1   class Concrete(LessAbstract): def bar(self): return 2 I would argue that instantiation is exactly when a concreteness failure should be recognized – although that could be via static checker and not at runtime.
Hide
Jim Bosch added a comment -

I would argue that instantiation is exactly when a concreteness failure should be recognized

, and if this wasn't Jira, also :100: and :facepalm:.

Show
Jim Bosch added a comment - I would argue that instantiation is exactly when a concreteness failure should be recognized , and if this wasn't Jira, also :100: and :facepalm:.
Hide
Nate Lust added a comment -

Kian-Tat Lim you are exactly correct. As much as I dislike the style of building up interfaces, I can understand people using it. And as much as I would like import time to be like compile time, its really not. So without some kind of finalize marker, import time just cant offer the kind of protection against this unless we disallow this pattern.

Not quite on topic, but this has convinced me even more that trying to do relations through abstract type inheritance just does not map well to python in a way that is 'pythonic' and that we should really just lean into mypy, typing, and protocols more (with various runtime checking in addition to import time).

Show
Nate Lust added a comment - Kian-Tat Lim you are exactly correct. As much as I dislike the style of building up interfaces, I can understand people using it. And as much as I would like import time to be like compile time, its really not. So without some kind of finalize marker, import time just cant offer the kind of protection against this unless we disallow this pattern. Not quite on topic, but this has convinced me even more that trying to do relations through abstract type inheritance just does not map well to python in a way that is 'pythonic' and that we should really just lean into mypy, typing, and protocols more (with various runtime checking in addition to import time).
Hide
Jim Bosch added a comment -

A number of people have brought up regular (instance) abstract properties here, and a Slack discussion made me realize that I was probably too quick to dismiss those as not relevant for this RFC: there are occasions where we do need to define an interface on the type, and for those we can't use a regular property and that's what I had intended to focus on.  But we often define an interface in terms of class attributes when we don't need to, because we don't actually need to get the values from a type object.  So that's probably the first thing any dev-guide text that comes from this RFC should say: if you find yourself wanting a derived class to provide an attribute to satisfy some interface, first ask whether the user of that class needs to access it from the type, and if they don't, it should be a regular base class instance attribute, an abstract regular property, or a regular abstract method.

Show
Jim Bosch added a comment - A number of people have brought up regular (instance) abstract properties here, and a Slack discussion made me realize that I was probably too quick to dismiss those as not relevant for this RFC: there are occasions where we do need to define an interface on the type, and for those we can't use a regular property and that's what I had intended to focus on.  But we often define an interface in terms of class attributes when we don't need to , because we don't actually need to get the values from a type object.  So that's probably the first thing any dev-guide text that comes from this RFC should say: if you find yourself wanting a derived class to provide an attribute to satisfy some interface, first ask whether the user of that class needs to access it from the type, and if they don't, it should be a regular base class instance attribute, an abstract regular property, or a regular abstract method.
Hide
Jim Bosch added a comment -

It's been a while since my last summarization post, with no comments since, so I think this is ready to adopt: to take into account previous comments, I'll recommend regular abstract methods in all cases where the user of an interface only works on instances, not subclasses, with classmethods only when needed.

Show
Jim Bosch added a comment - It's been a while since my last summarization post, with no comments since, so I think this is ready to adopt: to take into account previous comments, I'll recommend regular abstract methods in all cases where the user of an interface only works on instances, not subclasses, with classmethods only when needed.
Hide
Jim Bosch added a comment -

I should add that I'm not going to jump into implementing this immediately, as it's the holidays and I want to give watchers a chance to yell if they disagree; writing an implementation ticket and adopting this was just something I wanted to cross off my to-do list.

Show
Jim Bosch added a comment - I should add that I'm not going to jump into implementing this immediately, as it's the holidays and I want to give watchers a chance to yell if they disagree; writing an implementation ticket and adopting this was just something I wanted to cross off my to-do list.
Hide
John Parejko added a comment -

Please mention in your dev guide update a note like your comment from November: most use cases should just be instance attributes.

Show
John Parejko added a comment - Please mention in your dev guide update a note like your comment from November: most use cases should just be instance attributes.
Hide
Jim Bosch added a comment -

To be clear, I actually think most of these cases are best dealt with via an abstract instance method or property, and a base class that demands its subclasses define an instance attribute is something I would consider an antipattern. (A base class that defines its own instance attribute that isn't expected to be overridden is fine, too, but isn't really a customization hook.)

Show
Jim Bosch added a comment - To be clear, I actually think most of these cases are best dealt with via an abstract instance method or property, and a base class that demands its subclasses define an instance attribute is something I would consider an antipattern. (A base class that defines its own instance attribute that isn't expected to be overridden is fine, too, but isn't really a customization hook.)

#### People

Assignee:
Jim Bosch
Reporter:
Jim Bosch
Watchers:
Fred Moolekamp, Jim Bosch, John Parejko, Jonathan Sick, Kian-Tat Lim, Nate Lust, Tim Jenness